For this project you will take the role of a consultant hired by a real estate investment firm in Ames, Iowa, a mid-west town in the United States, to analyze data in order to help provide insight into how the firm should invest for highest profits, and to quantify and communicate to the company management what types of real estate properties are good investments and why. They have provided you with data on housing sales from between 2006 to 2010 that contains information about the characteristics of the house (number of bedrooms, number of bathrooms, square footage, etc.) and the house’s sale price. The codebook for this data set is available online here or in the Data folder in your repo.

About the Data Analysis Project

It’s generally a bad idea to buy the most expensive house in the neighborhood. And remember the real estate agents’ mantra: Location, location, location! Keep in mind that the goal is to make money for your investors, and hence investing in a property that is overvalued (costing more than it is worth) is rarely a good idea. This means that it’s critical to know which properties are overvalued and which are undervalued. The company that hired you has many questions for you about the housing market. It is up to you to decide what methods you want to use (frequentist or Bayesian) to answer these questions, and implement them to help to identify undervalued and overvalued properties.

You will have three data sets: a subset for training, a subset for testing, and a third subset for validation. You will be asked to do data exploration and build your model (or models) initially using only the training data. Then, you will test your model on the testing data, and finally validate using the validation data. We are challenging you to keep your analysis experience realistic, and in a realistic scenario you would not have access to all three of these data sets at once. You will be able to see on our scoreboard how well your team is doing based on its predictive performance on the testing data. After your project is turned in you will see the final score on the validation set.

All members of the team should contribute equally and answer any questions about the analysis at the final presentation.

For your analysis create a new notebook named “project.Rmd” and update accordingly rather than editing this.

Read in Training Data

To get started read in the training data.

library(dplyr)
library(tidyr)
load("ames_train.Rdata")

print(paste0("The dataset has ", dim(ames_train)[1], " number of observations and ", dim(ames_train)[2], " features"))

#Variables with NA's and their proportion of missing data
miss = apply(is.na(ames_train), 2, sum)
miss_prop = round(miss[miss>0]/nrow(ames_train), 3)
print(miss_prop) 
which(miss_prop>0.5) # four features have greater than 50% of data "missing" -- drop these variables

Notes about data cleaning:

We dropped “utilities” (type of utilities available) since in the training set, only 2 observations did not have all the utilities (electricity, gas, water and sewage). Intuitively, most modern property are equipped with these basic public utilities and keeping the variable would therefore be unnecessary.

We also dropped “condition 2” (proximity to various conditions if more than one is present) since it seemed redundant from our training set. Given Condition 1, only 12 properties were not close to normal conditions.

In the original scale, 1990 and 1900 would not be much different. Therefore, we changed the scale to the number of years since last construction or remodelling, subtracted from year 2010 (the end year in the dataset).

Another variable dropped in our model was “roof material” since only 1% of the property used material other than “standard composite shingle”. Similarly, “heating” was also dropped since more than 95% of the property has gas forced warm air furnace (GasA) instead of other types of heating.

For “exterior quality” and “exterior condition” , we recoded these ordinal variables to 1-5 to replace the original scale of conditions (from poor to excellent). Similarly, we recoded “basement exposure” and “basement rating” except that the new scale would start from 0 for properties without basement.

Exter Qual (Ordinal): Evaluates the quality of the material on the exterior

  Ex       Excellent
   Gd       Good
   TA       Average/Typical
   Fa       Fair
   Po       Poor

Exter Cond (Ordinal): Evaluates the present condition of the material on the exterior

  Ex       Excellent
   Gd       Good
   TA       Average/Typical
   Fa       Fair
   Po       Poor

Continuous variables such as “1st floor square feet” and “2nd floor square feet” were log-transformed for interpretation purpose.

For variable “functional”, we recoded different ordinal levels into binary levels — typical functionality or not, including minor and major deductions.

We summed up the number of bathrooms to one continuous variable. Note that one half-bathroom would be coded as 0.5.

# Did not remove any NA entries in Lot.frontage
data=ames_train
data <- data  %>%
  #filter(!is.na(Lot.Frontage)) %>% 
  mutate(MS.SubClass= factor(MS.SubClass)) %>%
  mutate(Alley =  factor(Alley, levels = levels(addNA(Alley)), labels = c(levels(Alley), "None"), exclude = NULL)) %>%
  mutate(HouseAge = Yr.Sold- pmax(Year.Built, Year.Remod.Add))  %>%
  filter(!is.na(Mas.Vnr.Area))  %>% 
  mutate(Bsmt.YN = 1*(!is.na(Bsmt.Qual))) %>%
  mutate(Bsmt.Qual =  factor(Bsmt.Qual, levels = levels(addNA(Bsmt.Qual)), labels = c(levels(Bsmt.Qual), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Qual = relevel(Bsmt.Qual, ref="None")) %>%
  mutate(Bsmt.Cond =  factor(Bsmt.Cond, levels = levels(addNA(Bsmt.Cond)), labels = c(levels(Bsmt.Cond), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Cond = relevel(Bsmt.Cond, ref="None")) %>%
  mutate(Bsmt.Exposure =  factor(Bsmt.Exposure, levels = levels(addNA(Bsmt.Exposure)), labels = c(levels(Bsmt.Exposure), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Exposure = relevel(Bsmt.Exposure, ref="None")) %>%
  mutate(BsmtFin.Type.1=  factor(BsmtFin.Type.1, levels = levels(addNA(BsmtFin.Type.1)), labels = c(levels(BsmtFin.Type.1), "None"), exclude = NULL)) %>%
  mutate(BsmtFin.Type.1 = relevel(BsmtFin.Type.1, ref="None")) %>%
  mutate(BsmtFin.Type.2=  factor(BsmtFin.Type.2, levels = levels(addNA(BsmtFin.Type.2)), labels = c(levels(BsmtFin.Type.2), "None"), exclude = NULL)) %>%
  mutate(BsmtFin.Type.2 = relevel(BsmtFin.Type.2, ref="None")) %>%
  mutate(X12.SF= X1st.Flr.SF+ X2nd.Flr.SF)  %>%   
  filter(!is.na(Bsmt.Full.Bath)) %>%
  filter(!is.na(Bsmt.Half.Bath)) %>%
  mutate(Baths = Bsmt.Full.Bath + 0.5*Bsmt.Half.Bath + Full.Bath + 0.5*Half.Bath) %>%
  mutate(Fireplace.YN = 1*(Fireplaces>0)) %>%
  mutate(Fireplace.Qu =  factor(Fireplace.Qu, levels = levels(addNA(Fireplace.Qu)), labels = c(levels(Fireplace.Qu), "None"), exclude = NULL)) %>%
  mutate(Fireplace.Qu = relevel(Fireplace.Qu, ref="None")) %>%
  mutate(Garage.YN = 1*(!is.na(Garage.Cond))) %>%
  mutate(Garage.Type =  factor(Garage.Type, levels = levels(addNA(Garage.Type)), labels = c(levels(Garage.Type), "None"), exclude = NULL)) %>%
  mutate(Garage.Type = relevel(Garage.Type, ref="None")) %>%
  mutate(Garage.Finish =  factor(Garage.Finish, levels = levels(addNA(Garage.Finish)), labels = c(levels(Garage.Finish), "None"), exclude = NULL)) %>%
  mutate(Garage.Finish = relevel(Garage.Finish, ref="None")) %>%
  mutate(Garage.Qual =  factor(Garage.Qual, levels = levels(addNA(Garage.Qual)), labels = c(levels(Garage.Qual), "None"), exclude = NULL)) %>%
  mutate(Garage.Qual = relevel(Garage.Qual, ref="None")) %>%
  mutate(Garage.Cond =  factor(Garage.Cond, levels = levels(addNA(Garage.Cond)), labels = c(levels(Garage.Cond), "None"), exclude = NULL)) %>%
  mutate(Garage.Cond = relevel(Garage.Cond, ref="None")) %>%
  mutate(Porch.Area = Wood.Deck.SF+ Open.Porch.SF+Enclosed.Porch+X3Ssn.Porch + Screen.Porch) %>%
  mutate(Pool.YN = 1*(Pool.Area>0)) %>%
  mutate(Pool.QC =  factor(Pool.QC, levels = levels(addNA(Pool.QC)), labels = c(levels(Pool.QC), "None"), exclude = NULL)) %>%
  mutate(Pool.QC = relevel(Pool.QC, ref="None")) %>%
  mutate(Fence =  factor(Fence, levels = levels(addNA(Fence)), labels = c(levels(Fence), "None"), exclude = NULL)) %>%
  mutate(Misc.Feature =  factor(Misc.Feature, levels = levels(addNA(Misc.Feature)), labels = c(levels(Misc.Feature), "None"), exclude = NULL)) %>%
  mutate(Mo.Sold = as.factor(Mo.Sold)) %>%
  mutate(Yr.Sold = as.factor(Yr.Sold)) %>%
  dplyr::select(-Garage.Yr.Blt) %>%
  mutate(Condition.1 = as.character(Condition.1)) %>%
  mutate(Kitchen.Qual=plyr::mapvalues(Kitchen.Qual, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
  mutate(Kitchen.Qual = as.numeric(as.character(Kitchen.Qual))) %>%
  mutate(Heating.QC=plyr::mapvalues(Heating.QC, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
  mutate(Heating.QC = as.numeric(as.character(Heating.QC))) %>%
  mutate(Bsmt.Qual = droplevels(Bsmt.Qual)) %>%
  mutate(Functional = droplevels(Functional)) %>%
  mutate(Roof.Matl = droplevels(Roof.Matl))
# Simplify Condition 1 (Park, Rail, Normal)
ind_rail<-which(data$Condition.1=="RRNn" | data$Condition.1=="RRAn" | data$Condition.1=="RRNe" | data$Condition.1=="RRAe")
ind_park<-which(data$Condition.1=="PosN" | data$Condition.1=="PosA")
data$Condition.1[ind_rail]<-"Rail"
data$Condition.1[ind_park]<-"Park"
data = data %>% 
  mutate(Condition.1 = factor(Condition.1)) %>%
  mutate(Condition.1 = relevel(Condition.1, ref="Norm")) 
# Eliminate the one entry in 'Exposure' that had been left completely empty
data_train<-data
data_train$Bsmt.Exposure[which(data_train$Bsmt.Exposure=="")]<-"None"
data_train$Bsmt.Exposure<-droplevels(data_train$Bsmt.Exposure)
data_train$Pool.Area<-data_train$Pool.Area+1
data_train$Total.Bsmt.SF<-data_train$Total.Bsmt.SF+1

The Neighborhood variable, typically of little interest other than to model the location effect, may be of more relevance when used with the map.

We are restricting attention to just the “normal sales” condition.

Part I: Simple Model

In the first model you are allowed only limited manipulations of the original data set to predict the sales price price. You are allowed to take power transformations of the original variables [square roots, logs, inverses, squares, etc.] but you are NOT allowed to create interaction variables. This means that a variable may only be used once in an equation [if you use $ x^2$ don’t use \(x\)]. Additionally, you may eliminate any data points you deem unfit. This model should have a minimum r-square of 73% (in the original units) and contain at least 6 variables but fewer than 20.

### perfromance evlaution function
performance<- function(Y, Yhat){
  
  bias<- mean(Y-Yhat[,1])
  
  max.dev<-max(abs(Y-Yhat[,1]))
  
  mean.dev<-mean(abs(Y-Yhat[,1]))
  
  RMSE<-sqrt(mean((Y-Yhat[,1])^2))
  
  coverage<-mean((Y>Yhat[,2]) & (Y<Yhat[,3]))
  
  out<-data.frame(bias=bias, max.dev=max.dev, mean.dev=mean.dev, RMES=RMSE, Coverage=coverage)
  
  return(out)
}
library(MASS)
# Base model with transformed predictors 
 model=lm(price ~ MS.SubClass + MS.Zoning + log(Lot.Frontage) + log(Lot.Area) +  Street + Alley + Lot.Shape + Land.Contour + Lot.Config + Land.Slope + Neighborhood + Condition.1  + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + HouseAge +   Roof.Style + Roof.Matl + Exterior.1st + Mas.Vnr.Type + log(1+Mas.Vnr.Area) + Exter.Cond + Exter.Qual +   Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + Total.Bsmt.SF + Heating + Heating.QC + Central.Air + Electrical + log(X12.SF) + log(1+Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type + Garage.Finish + Garage.Cars + Garage.Cond + Garage.Qual + Paved.Drive + log(1+Pool.Area) + Pool.QC + Fence + Misc.Val + Mo.Sold  +Yr.Sold + Sale.Type  + TotalSq, data=data_train)
# Boxcox (indicates that log is decent)
l<-boxcox(model)

expo<-round(l$x[which.max(l$y)],2)
## Current model
## log(Lot.Frontage) currently removed to have more data points (was not included when left in the model with BIC)
model.0=lm(log(price) ~ MS.SubClass + MS.Zoning  + log(Lot.Area) +  Street + Alley + Lot.Shape + Land.Contour + Lot.Config + Land.Slope + Neighborhood + Condition.1  + Bldg.Type + House.Style + Overall.Qual + Overall.Cond + HouseAge +   Roof.Style + Roof.Matl + Exterior.1st +  Mas.Vnr.Type + log(1+Mas.Vnr.Area) + Exter.Cond + Exter.Qual +   Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + log(Total.Bsmt.SF) + Bsmt.YN+ Heating + Heating.QC + Central.Air  + log(1+Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type + Garage.Finish + Garage.Cars +Garage.Qual+Garage.Cond  + Paved.Drive + log(Pool.Area) + Pool.QC + Fence + Misc.Val + Mo.Sold  +Yr.Sold + Sale.Type  + log(TotalSq) + Pool.YN , data=data_train)
# There are some perfect collinearities in this model -> eliminate it via AIC/BIC
summary(model.0)

Call:
lm(formula = log(price) ~ MS.SubClass + MS.Zoning + log(Lot.Area) + 
    Street + Alley + Lot.Shape + Land.Contour + Lot.Config + 
    Land.Slope + Neighborhood + Condition.1 + Bldg.Type + House.Style + 
    Overall.Qual + Overall.Cond + HouseAge + Roof.Style + Roof.Matl + 
    Exterior.1st + Mas.Vnr.Type + log(1 + Mas.Vnr.Area) + Exter.Cond + 
    Exter.Qual + Foundation + Bsmt.Qual + Bsmt.Cond + Bsmt.Exposure + 
    log(Total.Bsmt.SF) + Bsmt.YN + Heating + Heating.QC + Central.Air + 
    log(1 + Low.Qual.Fin.SF) + Baths + Bedroom.AbvGr + Kitchen.AbvGr + 
    Kitchen.Qual + Functional + Fireplaces + Fireplace.Qu + Garage.Type + 
    Garage.Finish + Garage.Cars + Garage.Qual + Garage.Cond + 
    Paved.Drive + log(Pool.Area) + Pool.QC + Fence + Misc.Val + 
    Mo.Sold + Yr.Sold + Sale.Type + log(TotalSq) + Pool.YN, data = data_train)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.43874 -0.04567  0.00024  0.04925  0.25557 

Coefficients: (8 not defined because of singularities)
                           Estimate Std. Error t value Pr(>|t|)    
(Intercept)               7.135e+00  2.287e-01  31.199  < 2e-16 ***
MS.SubClass30            -7.720e-02  1.621e-02  -4.762 2.13e-06 ***
MS.SubClass40            -2.625e-02  5.441e-02  -0.483 0.629528    
MS.SubClass45            -1.585e-01  1.216e-01  -1.304 0.192564    
MS.SubClass50            -2.953e-02  3.292e-02  -0.897 0.369868    
MS.SubClass60            -3.433e-02  2.747e-02  -1.250 0.211688    
MS.SubClass70            -7.417e-02  2.987e-02  -2.483 0.013157 *  
MS.SubClass75            -1.948e-02  8.094e-02  -0.241 0.809826    
MS.SubClass80            -6.554e-02  4.932e-02  -1.329 0.184105    
MS.SubClass85            -3.716e-02  3.572e-02  -1.040 0.298403    
MS.SubClass90            -2.750e-02  2.873e-02  -0.957 0.338788    
MS.SubClass120            3.104e-02  4.763e-02   0.652 0.514768    
MS.SubClass150           -1.220e-01  1.210e-01  -1.008 0.313770    
MS.SubClass160           -5.059e-02  5.922e-02  -0.854 0.393087    
MS.SubClass180           -2.540e-02  8.118e-02  -0.313 0.754415    
MS.SubClass190           -7.996e-02  9.993e-02  -0.800 0.423747    
MS.ZoningC (all)         -8.623e-02  1.100e-01  -0.784 0.433200    
MS.ZoningFV               1.407e-01  1.042e-01   1.351 0.176927    
MS.ZoningI (all)         -2.767e-02  1.291e-01  -0.214 0.830289    
MS.ZoningRH               1.659e-01  1.037e-01   1.599 0.110075    
MS.ZoningRL               1.231e-01  9.957e-02   1.236 0.216735    
MS.ZoningRM               7.809e-02  1.008e-01   0.775 0.438763    
log(Lot.Area)             9.002e-02  1.010e-02   8.912  < 2e-16 ***
StreetPave               -1.815e-02  4.429e-02  -0.410 0.682128    
AlleyPave                -1.134e-02  2.326e-02  -0.488 0.625946    
AlleyNone                 1.801e-02  1.395e-02   1.292 0.196747    
Lot.ShapeIR2             -6.732e-04  1.582e-02  -0.043 0.966069    
Lot.ShapeIR3              6.258e-03  3.092e-02   0.202 0.839656    
Lot.ShapeReg             -1.546e-03  5.964e-03  -0.259 0.795449    
Land.ContourHLS           2.953e-02  1.852e-02   1.594 0.111086    
Land.ContourLow          -5.651e-03  2.377e-02  -0.238 0.812139    
Land.ContourLvl           2.114e-02  1.375e-02   1.537 0.124571    
Lot.ConfigCulDSac         1.419e-02  1.163e-02   1.220 0.222861    
Lot.ConfigFR2            -3.046e-02  1.445e-02  -2.108 0.035210 *  
Lot.ConfigFR3            -2.514e-02  4.191e-02  -0.600 0.548733    
Lot.ConfigInside          1.273e-02  6.572e-03   1.937 0.053009 .  
Land.SlopeMod             1.605e-02  1.453e-02   1.104 0.269592    
Land.SlopeSev            -7.217e-02  5.346e-02  -1.350 0.177255    
NeighborhoodBlueste       1.012e-01  4.633e-02   2.184 0.029154 *  
NeighborhoodBrDale        1.193e-02  4.322e-02   0.276 0.782511    
NeighborhoodBrkSide       2.929e-02  3.587e-02   0.817 0.414264    
NeighborhoodClearCr       1.518e-02  3.742e-02   0.406 0.685092    
NeighborhoodCollgCr      -4.175e-03  3.026e-02  -0.138 0.890275    
NeighborhoodCrawfor       6.731e-02  3.434e-02   1.960 0.050179 .  
NeighborhoodEdwards      -8.750e-02  3.226e-02  -2.712 0.006778 ** 
NeighborhoodGilbert      -4.620e-02  3.155e-02  -1.464 0.143425    
NeighborhoodGreens        5.330e-02  4.491e-02   1.187 0.235555    
NeighborhoodGrnHill       4.587e-01  7.033e-02   6.522 9.94e-11 ***
NeighborhoodIDOTRR       -4.480e-02  3.976e-02  -1.127 0.260077    
NeighborhoodLandmrk      -8.096e-02  9.648e-02  -0.839 0.401562    
NeighborhoodMeadowV      -9.830e-02  4.989e-02  -1.970 0.049004 *  
NeighborhoodMitchel      -1.939e-02  3.247e-02  -0.597 0.550505    
NeighborhoodNAmes        -5.754e-02  3.165e-02  -1.818 0.069271 .  
NeighborhoodNoRidge       7.880e-02  3.380e-02   2.332 0.019881 *  
NeighborhoodNPkVill       4.515e-02  4.172e-02   1.082 0.279425    
NeighborhoodNridgHt       3.993e-02  3.232e-02   1.235 0.216929    
NeighborhoodNWAmes       -3.705e-02  3.264e-02  -1.135 0.256547    
NeighborhoodOldTown      -5.117e-02  3.647e-02  -1.403 0.160835    
NeighborhoodSawyer       -1.697e-02  3.244e-02  -0.523 0.600886    
NeighborhoodSawyerW      -3.472e-02  3.147e-02  -1.103 0.270103    
NeighborhoodSomerst       7.220e-02  4.009e-02   1.801 0.071939 .  
NeighborhoodStoneBr       7.764e-02  3.451e-02   2.250 0.024620 *  
NeighborhoodSWISU        -4.571e-02  3.748e-02  -1.220 0.222869    
NeighborhoodTimber       -2.846e-02  3.383e-02  -0.841 0.400336    
NeighborhoodVeenker       2.246e-02  3.973e-02   0.565 0.571929    
Condition.1Artery        -7.044e-02  1.535e-02  -4.589 4.90e-06 ***
Condition.1Feedr         -7.402e-02  1.141e-02  -6.486 1.25e-10 ***
Condition.1Park          -2.670e-04  1.739e-02  -0.015 0.987752    
Condition.1Rail          -5.997e-02  1.480e-02  -4.052 5.38e-05 ***
Bldg.Type2fmCon           3.790e-02  9.699e-02   0.391 0.696031    
Bldg.TypeDuplex                  NA         NA      NA       NA    
Bldg.TypeTwnhs           -4.101e-02  4.983e-02  -0.823 0.410662    
Bldg.TypeTwnhsE          -3.401e-02  4.657e-02  -0.730 0.465340    
House.Style1.5Unf         1.452e-01  1.200e-01   1.210 0.226482    
House.Style1Story         4.451e-02  3.137e-02   1.419 0.156180    
House.Style2.5Fin        -8.497e-02  9.804e-02  -0.867 0.386254    
House.Style2.5Unf         4.834e-03  8.411e-02   0.057 0.954181    
House.Style2Story         4.421e-02  2.947e-02   1.500 0.133822    
House.StyleSFoyer         9.402e-02  4.126e-02   2.279 0.022844 *  
House.StyleSLvl           1.099e-01  5.280e-02   2.082 0.037563 *  
Overall.Qual              4.386e-02  3.647e-03  12.025  < 2e-16 ***
Overall.Cond              3.438e-02  3.001e-03  11.455  < 2e-16 ***
HouseAge                 -5.985e-04  1.928e-04  -3.104 0.001954 ** 
Roof.StyleGable          -6.870e-02  5.100e-02  -1.347 0.178219    
Roof.StyleGambrel        -1.341e-01  5.823e-02  -2.303 0.021454 *  
Roof.StyleHip            -7.569e-02  5.137e-02  -1.473 0.140909    
Roof.StyleMansard        -9.772e-02  6.256e-02  -1.562 0.118568    
Roof.StyleShed           -1.860e-02  9.228e-02  -0.202 0.840325    
Roof.MatlMembran          8.715e-02  1.139e-01   0.765 0.444332    
Roof.MatlRoll             7.002e-02  9.408e-02   0.744 0.456888    
Roof.MatlTar&Grv          3.100e-02  3.754e-02   0.826 0.408998    
Roof.MatlWdShake          1.155e-02  4.850e-02   0.238 0.811768    
Roof.MatlWdShngl          7.904e-02  5.041e-02   1.568 0.117148    
Exterior.1stAsphShn      -3.774e-02  9.304e-02  -0.406 0.685059    
Exterior.1stBrkComm       1.197e-01  7.003e-02   1.710 0.087573 .  
Exterior.1stBrkFace       6.513e-02  2.716e-02   2.398 0.016616 *  
Exterior.1stCBlock               NA         NA      NA       NA    
Exterior.1stCemntBd       4.768e-02  2.816e-02   1.693 0.090731 .  
Exterior.1stHdBoard       1.592e-02  2.409e-02   0.661 0.508939    
Exterior.1stImStucc      -1.087e-02  9.163e-02  -0.119 0.905605    
Exterior.1stMetalSd       2.877e-02  2.355e-02   1.222 0.222118    
Exterior.1stPlywood       9.545e-03  2.532e-02   0.377 0.706277    
Exterior.1stPreCast       3.289e-01  1.013e-01   3.245 0.001204 ** 
Exterior.1stStucco        1.632e-02  3.006e-02   0.543 0.587327    
Exterior.1stVinylSd       3.341e-02  2.395e-02   1.395 0.163211    
Exterior.1stWd Sdng       1.428e-02  2.354e-02   0.607 0.544233    
Exterior.1stWdShing       2.649e-02  2.844e-02   0.932 0.351759    
Mas.Vnr.TypeBrkFace       5.390e-02  2.675e-02   2.015 0.044108 *  
Mas.Vnr.TypeNone          1.103e-01  3.638e-02   3.031 0.002488 ** 
Mas.Vnr.TypeStone         5.822e-02  2.849e-02   2.044 0.041189 *  
log(1 + Mas.Vnr.Area)     1.221e-02  5.113e-03   2.387 0.017131 *  
Exter.CondFa             -2.636e-02  4.346e-02  -0.606 0.544314    
Exter.CondGd              2.375e-02  3.777e-02   0.629 0.529609    
Exter.CondPo             -1.132e-01  1.012e-01  -1.119 0.263270    
Exter.CondTA              3.107e-02  3.777e-02   0.823 0.410846    
Exter.QualFa             -5.306e-02  3.688e-02  -1.439 0.150478    
Exter.QualGd             -5.349e-02  1.970e-02  -2.715 0.006716 ** 
Exter.QualTA             -5.260e-02  2.179e-02  -2.414 0.015904 *  
FoundationCBlock          2.047e-02  1.118e-02   1.831 0.067407 .  
FoundationPConc           4.603e-02  1.188e-02   3.873 0.000113 ***
FoundationSlab           -6.749e-03  3.072e-02  -0.220 0.826142    
FoundationStone           4.719e-02  4.164e-02   1.133 0.257252    
FoundationWood            5.419e-02  5.410e-02   1.002 0.316657    
Bsmt.QualEx              -5.934e-01  1.230e-01  -4.826 1.56e-06 ***
Bsmt.QualFa              -6.228e-01  1.225e-01  -5.084 4.25e-07 ***
Bsmt.QualGd              -6.587e-01  1.224e-01  -5.383 8.69e-08 ***
Bsmt.QualPo              -1.649e-01  1.733e-01  -0.952 0.341390    
Bsmt.QualTA              -6.586e-01  1.221e-01  -5.395 8.14e-08 ***
Bsmt.CondEx              -5.374e-04  6.234e-02  -0.009 0.993124    
Bsmt.CondFa              -2.399e-02  1.472e-02  -1.630 0.103428    
Bsmt.CondGd               1.951e-03  1.303e-02   0.150 0.881017    
Bsmt.CondPo               8.582e-02  7.715e-02   1.112 0.266197    
Bsmt.CondTA                      NA         NA      NA       NA    
Bsmt.ExposureAv           1.326e-01  8.588e-02   1.544 0.122909    
Bsmt.ExposureGd           1.633e-01  8.631e-02   1.892 0.058769 .  
Bsmt.ExposureMn           9.901e-02  8.612e-02   1.150 0.250533    
Bsmt.ExposureNo           1.089e-01  8.585e-02   1.269 0.204848    
log(Total.Bsmt.SF)        9.271e-02  1.217e-02   7.616 5.06e-14 ***
Bsmt.YN                          NA         NA      NA       NA    
HeatingGasA               7.688e-02  9.347e-02   0.823 0.410924    
HeatingGasW               1.523e-01  9.678e-02   1.574 0.115838    
HeatingGrav               1.256e-02  1.146e-01   0.110 0.912736    
HeatingOthW               3.011e-02  1.146e-01   0.263 0.792809    
HeatingWall               1.054e-01  1.038e-01   1.015 0.310188    
Heating.QC                8.207e-03  3.392e-03   2.419 0.015689 *  
Central.AirY              5.164e-02  1.337e-02   3.862 0.000118 ***
log(1 + Low.Qual.Fin.SF)  1.195e-02  4.013e-03   2.977 0.002968 ** 
Baths                     4.982e-02  4.577e-03  10.884  < 2e-16 ***
Bedroom.AbvGr            -1.038e-02  4.588e-03  -2.263 0.023781 *  
Kitchen.AbvGr            -1.030e-01  2.454e-02  -4.199 2.87e-05 ***
Kitchen.Qual              3.426e-02  5.840e-03   5.866 5.66e-09 ***
FunctionalMaj2           -2.494e-01  5.193e-02  -4.804 1.74e-06 ***
FunctionalMin1           -1.625e-02  3.566e-02  -0.456 0.648641    
FunctionalMin2           -1.274e-02  3.606e-02  -0.353 0.723946    
FunctionalMod            -3.512e-02  3.942e-02  -0.891 0.373152    
FunctionalTyp             4.746e-02  3.290e-02   1.443 0.149399    
Fireplaces                2.516e-02  8.781e-03   2.865 0.004240 ** 
Fireplace.QuEx            1.185e-02  2.345e-02   0.505 0.613479    
Fireplace.QuFa           -3.809e-03  1.740e-02  -0.219 0.826753    
Fireplace.QuGd            8.029e-03  1.221e-02   0.658 0.510907    
Fireplace.QuPo            1.114e-02  2.043e-02   0.545 0.585585    
Fireplace.QuTA            1.034e-03  1.211e-02   0.085 0.931945    
Garage.Type2Types        -2.409e-02  3.354e-02  -0.718 0.472872    
Garage.TypeAttchd         7.027e-03  1.677e-02   0.419 0.675292    
Garage.TypeBasment       -1.302e-02  3.140e-02  -0.415 0.678518    
Garage.TypeBuiltIn        1.970e-02  1.991e-02   0.989 0.322625    
Garage.TypeCarPort       -7.919e-02  5.498e-02  -1.440 0.150039    
Garage.TypeDetchd         1.634e-02  1.637e-02   0.998 0.318665    
Garage.Finish            -7.485e-02  1.519e-01  -0.493 0.622210    
Garage.FinishFin          1.826e-02  8.378e-03   2.180 0.029443 *  
Garage.FinishRFn         -5.550e-03  7.428e-03  -0.747 0.455052    
Garage.FinishUnf                 NA         NA      NA       NA    
Garage.Cars               3.427e-02  5.610e-03   6.109 1.33e-09 ***
Garage.QualFa            -1.092e-02  1.439e-02  -0.759 0.448201    
Garage.QualGd             4.510e-02  2.910e-02   1.550 0.121430    
Garage.QualPo            -3.117e-01  7.767e-02  -4.013 6.35e-05 ***
Garage.QualTA                    NA         NA      NA       NA    
Garage.CondFa            -6.362e-02  1.804e-02  -3.527 0.000435 ***
Garage.CondGd             6.926e-04  4.160e-02   0.017 0.986719    
Garage.CondPo             7.258e-02  4.399e-02   1.650 0.099151 .  
Garage.CondTA                    NA         NA      NA       NA    
Paved.DriveP             -1.649e-02  1.865e-02  -0.884 0.376870    
Paved.DriveY              2.848e-02  1.229e-02   2.318 0.020627 *  
log(Pool.Area)           -2.200e-01  2.224e-01  -0.989 0.322688    
Pool.QCEx                 1.317e+00  1.152e+00   1.143 0.253103    
Pool.QCFa                 1.404e+00  1.416e+00   0.991 0.321709    
Pool.QCGd                 1.446e+00  1.447e+00   0.999 0.317933    
Pool.QCTA                 1.684e+00  1.362e+00   1.236 0.216777    
FenceGdWo                 1.227e-03  1.707e-02   0.072 0.942711    
FenceMnPrv               -1.599e-02  1.353e-02  -1.182 0.237505    
FenceMnWw                -4.282e-02  3.570e-02  -1.199 0.230655    
FenceNone                -1.278e-02  1.234e-02  -1.036 0.300596    
Misc.Val                  8.128e-07  4.966e-06   0.164 0.870024    
Mo.Sold2                 -9.601e-03  1.705e-02  -0.563 0.573370    
Mo.Sold3                 -1.316e-02  1.525e-02  -0.863 0.388463    
Mo.Sold4                  1.720e-02  1.479e-02   1.163 0.245173    
Mo.Sold5                  1.143e-02  1.415e-02   0.808 0.419406    
Mo.Sold6                  1.087e-02  1.386e-02   0.785 0.432847    
Mo.Sold7                  1.317e-02  1.400e-02   0.941 0.347126    
Mo.Sold8                  1.980e-03  1.537e-02   0.129 0.897511    
 [ reached getOption("max.print") -- omitted 18 rows ]
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.08396 on 1283 degrees of freedom
Multiple R-squared:  0.9562,    Adjusted R-squared:  0.9491 
F-statistic:   134 on 209 and 1283 DF,  p-value: < 2.2e-16
plot(model.0)
not plotting observations with leverage one:
  47, 168, 242, 284, 411, 638, 655, 739, 804, 990, 1011, 1194, 1198, 1324, 1344, 1376

not plotting observations with leverage one:
  47, 168, 242, 284, 411, 638, 655, 739, 804, 990, 1011, 1194, 1198, 1324, 1344, 1376

NaNs producedNaNs produced

#AIC
#model.AIC=step(model.0, k=2)
#summary(model.AIC)
#plot(model.AIC)
#BIC
#model.BIC=step(model.0, k=log(nrow(data_train)))
model.BIC=lm(formula = log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
               Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
               Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
               Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
               Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
               log(TotalSq), data = data_train)
summary(model.BIC)

Call:
lm(formula = log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
    Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
    Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
    Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
    Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
    log(TotalSq), data = data_train)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.66831 -0.05150  0.00025  0.05629  0.32251 

Coefficients:
                      Estimate Std. Error t value Pr(>|t|)    
(Intercept)          7.3926554  0.1169000  63.239  < 2e-16 ***
log(Lot.Area)        0.1026436  0.0075547  13.587  < 2e-16 ***
NeighborhoodBlueste  0.0146631  0.0419500   0.350 0.726738    
NeighborhoodBrDale  -0.0634618  0.0347412  -1.827 0.067954 .  
NeighborhoodBrkSide -0.0219612  0.0306986  -0.715 0.474491    
NeighborhoodClearCr  0.0028061  0.0338084   0.083 0.933862    
NeighborhoodCollgCr -0.0131967  0.0275606  -0.479 0.632136    
NeighborhoodCrawfor  0.0479177  0.0311068   1.540 0.123679    
NeighborhoodEdwards -0.1000400  0.0294309  -3.399 0.000695 ***
NeighborhoodGilbert -0.0379098  0.0287188  -1.320 0.187036    
NeighborhoodGreens   0.0359676  0.0442891   0.812 0.416864    
NeighborhoodGrnHill  0.4397167  0.0709210   6.200 7.38e-10 ***
NeighborhoodIDOTRR  -0.1451682  0.0324860  -4.469 8.49e-06 ***
NeighborhoodLandmrk -0.0681952  0.0949941  -0.718 0.472943    
NeighborhoodMeadowV -0.1266187  0.0383000  -3.306 0.000970 ***
NeighborhoodMitchel -0.0368471  0.0295088  -1.249 0.211987    
NeighborhoodNAmes   -0.0565614  0.0285542  -1.981 0.047801 *  
NeighborhoodNoRidge  0.0623060  0.0303150   2.055 0.040034 *  
NeighborhoodNPkVill -0.0208120  0.0384833  -0.541 0.588726    
NeighborhoodNridgHt  0.0509080  0.0290237   1.754 0.079643 .  
NeighborhoodNWAmes  -0.0544560  0.0297828  -1.828 0.067693 .  
NeighborhoodOldTown -0.1188641  0.0294370  -4.038 5.68e-05 ***
NeighborhoodSawyer  -0.0229563  0.0298927  -0.768 0.442639    
NeighborhoodSawyerW -0.0515292  0.0289758  -1.778 0.075559 .  
NeighborhoodSomerst  0.0566938  0.0276847   2.048 0.040759 *  
NeighborhoodStoneBr  0.0745779  0.0321385   2.321 0.020453 *  
NeighborhoodSWISU   -0.0604508  0.0340779  -1.774 0.076293 .  
NeighborhoodTimber  -0.0324647  0.0317804  -1.022 0.307176    
NeighborhoodVeenker -0.0167331  0.0376356  -0.445 0.656671    
Condition.1Artery   -0.0760394  0.0151294  -5.026 5.64e-07 ***
Condition.1Feedr    -0.0767356  0.0113456  -6.763 1.96e-11 ***
Condition.1Park      0.0069725  0.0182004   0.383 0.701706    
Condition.1Rail     -0.0496206  0.0149453  -3.320 0.000922 ***
Overall.Qual         0.0494652  0.0034280  14.430  < 2e-16 ***
Overall.Cond         0.0351455  0.0027881  12.605  < 2e-16 ***
HouseAge            -0.0008503  0.0001883  -4.516 6.84e-06 ***
FoundationCBlock     0.0583435  0.0105020   5.555 3.30e-08 ***
FoundationPConc      0.0718433  0.0115567   6.217 6.66e-10 ***
FoundationSlab       0.0630207  0.0291017   2.166 0.030512 *  
FoundationStone      0.0038723  0.0390916   0.099 0.921107    
FoundationWood       0.0465776  0.0554929   0.839 0.401418    
Bsmt.QualEx         -0.7570306  0.1136937  -6.659 3.94e-11 ***
Bsmt.QualFa         -0.8357544  0.1124732  -7.431 1.85e-13 ***
Bsmt.QualGd         -0.8407558  0.1123115  -7.486 1.24e-13 ***
Bsmt.QualPo         -0.8411315  0.1443930  -5.825 7.04e-09 ***
Bsmt.QualTA         -0.8503101  0.1122157  -7.577 6.31e-14 ***
Bsmt.ExposureAv      0.1590409  0.0918543   1.731 0.083588 .  
Bsmt.ExposureGd      0.1929930  0.0921545   2.094 0.036415 *  
Bsmt.ExposureMn      0.1151061  0.0920620   1.250 0.211392    
Bsmt.ExposureNo      0.1296663  0.0917945   1.413 0.158000    
log(Total.Bsmt.SF)   0.1191674  0.0091824  12.978  < 2e-16 ***
Heating.QC           0.0091548  0.0033638   2.722 0.006577 ** 
Central.AirY         0.0587416  0.0121548   4.833 1.49e-06 ***
Baths                0.0470497  0.0045251  10.397  < 2e-16 ***
Bedroom.AbvGr       -0.0140992  0.0042615  -3.308 0.000961 ***
Kitchen.AbvGr       -0.0977644  0.0140392  -6.964 5.05e-12 ***
Kitchen.Qual         0.0346078  0.0056027   6.177 8.51e-10 ***
FunctionalMaj2      -0.1661268  0.0518564  -3.204 0.001387 ** 
FunctionalMin1       0.0143539  0.0345741   0.415 0.678085    
FunctionalMin2       0.0352368  0.0342227   1.030 0.303356    
FunctionalMod       -0.0059146  0.0372751  -0.159 0.873948    
FunctionalTyp        0.0882264  0.0312921   2.819 0.004877 ** 
Fireplaces           0.0278606  0.0047452   5.871 5.37e-09 ***
Garage.Cars          0.0400643  0.0046975   8.529  < 2e-16 ***
Paved.DriveP         0.0101521  0.0181899   0.558 0.576854    
Paved.DriveY         0.0573398  0.0114111   5.025 5.67e-07 ***
log(Pool.Area)       0.0175609  0.0059007   2.976 0.002969 ** 
log(TotalSq)         0.3701184  0.0154033  24.029  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.09123 on 1425 degrees of freedom
Multiple R-squared:  0.9426,    Adjusted R-squared:  0.9399 
F-statistic:   349 on 67 and 1425 DF,  p-value: < 2.2e-16
plot(model.BIC)
not plotting observations with leverage one:
  638, 655, 990

not plotting observations with leverage one:
  638, 655, 990

# Exploring the remaining predictors relationship to price
# plot(log(data_train$Lot.Area), log(data_train$price))
# plot((data_train$Neighborhood), log(data_train$price))
# plot((data_train$Condition.1), log(data_train$price))
# plot((data_train$Overall.Qual), log(data_train$price))
# plot((data_train$Overall.Cond), log(data_train$price))
# plot((data_train$HouseAge), log(data_train$price))
# plot((data_train$Bsmt.Qual), log(data_train$price))
# plot((data_train$Bsmt.Exposure), log(data_train$price))
# plot(log(data_train$Total.Bsmt.SF), log(data_train$price))
# plot((data_train$Heating.QC), log(data_train$price))
# plot((data_train$Central.Air), log(data_train$price))
# plot((data_train$Baths), log(data_train$price))
# plot((data_train$Bedroom.AbvGr), log(data_train$price))
# plot((data_train$Kitchen.AbvGr), log(data_train$price))
# plot((data_train$Kitchen.Qual), log(data_train$price))
# plot((data_train$Functional), log(data_train$price))
# plot((data_train$Fireplaces), log(data_train$price))
# plot((data_train$Paved.Drive), log(data_train$price))
# plot((data_train$Garage.Cars), log(data_train$price))
# plot((data_train$Garage.Cars), log(data_train$price))
# plot(log(1+data_train$Pool.Area), log(data_train$price))
# plot(log(data_train$TotalSq), log(data_train$price))
termplot(model.BIC, partial.resid = TRUE, col.res = "purple", cex = 0.5,
         rug = T, se = T, smooth = panel.smooth)

# There are 3 high leverage points - may want to exclude them
hh<-hatvalues(model.BIC)
id<-which(hh==1)
plot(hatvalues(model.BIC), type = "h")

## Prepare test data
load("ames_test.Rdata")

data=ames_test

data <- data  %>%  
  #filter(!is.na(Lot.Frontage)) %>% 
  mutate(MS.SubClass= factor(MS.SubClass)) %>%
  mutate(Alley =  factor(Alley, levels = levels(addNA(Alley)), labels = c(levels(Alley), "None"), exclude = NULL)) %>%
  mutate(HouseAge = Yr.Sold- pmax(Year.Built, Year.Remod.Add))  %>%
  #filter(!is.na(Mas.Vnr.Area))  %>% 
  mutate(Bsmt.YN = 1*(!is.na(Bsmt.Qual))) %>%
  mutate(Bsmt.Qual =  factor(Bsmt.Qual, levels = levels(addNA(Bsmt.Qual)), labels = c(levels(Bsmt.Qual), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Qual = relevel(Bsmt.Qual, ref="None")) %>%
  mutate(Bsmt.Cond =  factor(Bsmt.Cond, levels = levels(addNA(Bsmt.Cond)), labels = c(levels(Bsmt.Cond), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Cond = relevel(Bsmt.Cond, ref="None")) %>%
  mutate(Bsmt.Exposure =  factor(Bsmt.Exposure, levels = levels(addNA(Bsmt.Exposure)), labels = c(levels(Bsmt.Exposure), "None"), exclude = NULL)) %>%
  mutate(Bsmt.Exposure = relevel(Bsmt.Exposure, ref="None")) %>%
  mutate(BsmtFin.Type.1=  factor(BsmtFin.Type.1, levels = levels(addNA(BsmtFin.Type.1)), labels = c(levels(BsmtFin.Type.1), "None"), exclude = NULL)) %>%
  mutate(BsmtFin.Type.1 = relevel(BsmtFin.Type.1, ref="None")) %>%
  mutate(BsmtFin.Type.2=  factor(BsmtFin.Type.2, levels = levels(addNA(BsmtFin.Type.2)), labels = c(levels(BsmtFin.Type.2), "None"), exclude = NULL)) %>%
  mutate(BsmtFin.Type.2 = relevel(BsmtFin.Type.2, ref="None")) %>%
  mutate(X12.SF= X1st.Flr.SF+ X2nd.Flr.SF)  %>%   
  #filter(!is.na(Bsmt.Full.Bath)) %>%
  #filter(!is.na(Bsmt.Half.Bath)) %>%
  mutate(Baths = Bsmt.Full.Bath + 0.5*Bsmt.Half.Bath + Full.Bath + 0.5*Half.Bath) %>%
  mutate(Fireplace.YN = 1*(Fireplaces>0)) %>%
  mutate(Fireplace.Qu =  factor(Fireplace.Qu, levels = levels(addNA(Fireplace.Qu)), labels = c(levels(Fireplace.Qu), "None"), exclude = NULL)) %>%
  mutate(Fireplace.Qu = relevel(Fireplace.Qu, ref="None")) %>%
  mutate(Garage.YN = 1*(!is.na(Garage.Cond))) %>%
  mutate(Garage.Type =  factor(Garage.Type, levels = levels(addNA(Garage.Type)), labels = c(levels(Garage.Type), "None"), exclude = NULL)) %>%
  mutate(Garage.Type = relevel(Garage.Type, ref="None")) %>%
  mutate(Garage.Finish =  factor(Garage.Finish, levels = levels(addNA(Garage.Finish)), labels = c(levels(Garage.Finish), "None"), exclude = NULL)) %>%
  mutate(Garage.Finish = relevel(Garage.Finish, ref="None")) %>%
  mutate(Garage.Qual =  factor(Garage.Qual, levels = levels(addNA(Garage.Qual)), labels = c(levels(Garage.Qual), "None"), exclude = NULL)) %>%
  mutate(Garage.Qual = relevel(Garage.Qual, ref="None")) %>%
  mutate(Garage.Cond =  factor(Garage.Cond, levels = levels(addNA(Garage.Cond)), labels = c(levels(Garage.Cond), "None"), exclude = NULL)) %>%
  mutate(Garage.Cond = relevel(Garage.Cond, ref="None")) %>%
  mutate(Porch.Area = Wood.Deck.SF+ Open.Porch.SF+Enclosed.Porch+X3Ssn.Porch + Screen.Porch) %>%
  mutate(Pool.YN = 1*(Pool.Area>0)) %>%
  mutate(Pool.QC =  factor(Pool.QC, levels = levels(addNA(Pool.QC)), labels = c(levels(Pool.QC), "None"), exclude = NULL)) %>%
  mutate(Pool.QC = relevel(Pool.QC, ref="None")) %>%
  mutate(Fence =  factor(Fence, levels = levels(addNA(Fence)), labels = c(levels(Fence), "None"), exclude = NULL)) %>%
  mutate(Misc.Feature =  factor(Misc.Feature, levels = levels(addNA(Misc.Feature)), labels = c(levels(Misc.Feature), "None"), exclude = NULL)) %>%
  mutate(Mo.Sold = as.factor(Mo.Sold)) %>%
  mutate(Yr.Sold = as.factor(Yr.Sold)) %>%
  dplyr::select(-Garage.Yr.Blt) %>%
  mutate(Condition.1 = as.character(Condition.1)) %>%
  mutate(Kitchen.Qual=plyr::mapvalues(Kitchen.Qual, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
  mutate(Kitchen.Qual = as.numeric(as.character(Kitchen.Qual))) %>%
  mutate(Heating.QC=plyr::mapvalues(Heating.QC, from = c("Po", "Fa", "TA","Gd", "Ex" ), to = c("1", "2", "3", "4", "5"))) %>%
  mutate(Heating.QC = as.numeric(as.character(Heating.QC))) %>%
  mutate(Bsmt.Qual = droplevels(Bsmt.Qual)) %>%
  mutate(Functional = droplevels(Functional)) %>%
  mutate(Roof.Matl = droplevels(Roof.Matl))

ind_rail<-which(data$Condition.1=="RRNn" | data$Condition.1=="RRAn" | data$Condition.1=="RRNe" | data$Condition.1=="RRAe")
ind_park<-which(data$Condition.1=="PosN" | data$Condition.1=="PosA")
data$Condition.1[ind_rail]<-"Rail"
data$Condition.1[ind_park]<-"Park"

data = data %>% 
  mutate(Condition.1 = factor(Condition.1)) %>%
  mutate(Condition.1 = relevel(Condition.1, ref="Norm")) 

data_test=data


data_test$Bsmt.Exposure[which(data_test$Bsmt.Exposure=="")]<-"None"
data_test$Bsmt.Exposure<-droplevels(data_test$Bsmt.Exposure)


data_test$Pool.Area<-data_test$Pool.Area+1
data_test$Total.Bsmt.SF<-data_test$Total.Bsmt.SF+1


# extract the truth
Y = data_test$price

# make prediction based on specific model
Yhat = predict(model.BIC, newdata=data_test, interval="predict")

# depending on the response transformation
Yhat = exp(Yhat)


# name dataframe as predictions! DO NOT CHANGE
predictions = as.data.frame(Yhat)
predictions$PID = data_test$PID

save(predictions, file="predict.Rdata")

performance(Y, Yhat)
# Lasso for the simplified model (from BIC)
library(glmnet)

X.train = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
               Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
               Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
               Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
               Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
               log(TotalSq), data = data_train)


X.test = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
               Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
               Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
               Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
               Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
               log(TotalSq), data = data_test)


model.lasso = glmnet(X.train, log(data_train$price), alpha=1)
cv.lasso = cv.glmnet(X.train, log(data_train$price), alpha=1)
yhat.lasso = exp(predict(model.lasso, s=cv.lasso$lambda.min, type="response", newx = X.test))

sqrt(mean(((yhat.lasso)-data_test$price)^2))


# Ridge for the simplified model (from BIC)

model.ridge = glmnet(X.train, log(data_train$price), alpha=0)
cv.ridge = cv.glmnet(X.train, log(data_train$price), alpha=0)
yhat.ridge = predict(model.ridge, s=cv.ridge$lambda.min, type="response", newx = X.test)

sqrt(mean((exp(yhat.ridge)-data_test$price)^2))
# BMA for simple model (from BIC)
library(BAS)
model.bma = bas.lm(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
                     Overall.Qual + Overall.Cond + HouseAge + Foundation + Bsmt.Qual + 
                     Bsmt.Exposure + log(Total.Bsmt.SF) + Heating.QC + Central.Air + 
                     Baths + Bedroom.AbvGr + Kitchen.AbvGr + Kitchen.Qual + Functional + 
                     Fireplaces + Garage.Cars + Paved.Drive + log(Pool.Area) + 
                     log(TotalSq), data=data_train,  prior="g-prior", a=nrow(data_train), modelprior=uniform(), method="MCMC", MCMC.iterations = 200000, thin = 20)
y_pred<-predict.bas(model.bma,newdata=data_test, estimator="BPM", prediction=TRUE, se.fit=TRUE)
y_hat<-exp(confint.pred.bas(y_pred))
y_hat= y_hat[,c(3,1,2)]
performance(data_test$price, y_hat)

library(treemap)

tree.data = data_train %>% mutate(price.bin = ifelse(price <= 50000, "<= $50,000", ifelse(price > 50000 & price <= 100000, "$50,000-100,000", ifelse(price > 100000 & price <= 200000, "$100,000-200,000", ifelse(price > 200000 & price <= 300000, "$200,000-300,000", ifelse(price > 300000 & price <= 400000, "$300,000-400,000", ifelse(price > 400000 & price <= 500000, "$400,000-500,000", "> $500,000")))))))

area.plot = treemap(tree.data, index="price.bin", vSize="Lot.Area", type="index", palette="YlGn", algorithm="pivotSize", title="Effect of Lot Area on Price Range", fontsize.title = 14)

data_train$Overall.Qual = as.numeric(data_train$Overall.Qual)
quality.plot = ggplot(train, aes(x = Overall.Qual, y=price)) + geom_smooth(color="gold3") + geom_count(color="red3") + theme_minimal()
plot(quality.plot)

Model Evaluation on Test Data

Create predicted values for price using your model using the testing data

load("ames_test.Rdata")

#Variables with NA's and their proportion of missing data
miss = apply(is.na(ames_train), 2, sum)
miss_prop = round(miss[miss>0]/nrow(ames_train), 3)
print(miss_prop) 
which(miss_prop>0.5) # four features have greater than 50% of data missing


#Created binary variables for whether or not a house as an alley, pool, fence, misc. feature, fireplace, basement or garage, as I thought it may be more meaningful- Tom
#May consider adding Lot.Frontage back in, as it only had 286 missing
#Have to decide between filtering out N/A's for different Garage variables and only including 'has.garage'
#Have to decide between filtering out N/A's for different Basement variables and only including 'has.basement'
test <- ames_test  %>%   filter(!is.na(Lot.Frontage)) %>% 
  mutate(Alley =  factor(Alley, levels = levels(addNA(Alley)), labels = c(levels(Alley), "None"), exclude = NULL)) %>%
  mutate(HouseAge = Yr.Sold- pmax(Year.Built, Year.Remod.Add))  %>%
  filter(!is.na(Mas.Vnr.Area))  %>% 
  mutate(Bsmt.YN = !is.na(Bsmt.Qual)) %>%
    mutate(Bsmt.Qual =  factor(Bsmt.Qual, levels = levels(addNA(Bsmt.Qual)), labels = c(levels(Bsmt.Qual), "NoBa"), exclude = NULL)) %>%
    mutate(Bsmt.Cond =  factor(Bsmt.Cond, levels = levels(addNA(Bsmt.Cond)), labels = c(levels(Bsmt.Cond), "NoBa"), exclude = NULL)) %>%
    mutate(Bsmt.Exposure =  factor(Bsmt.Exposure, levels = levels(addNA(Bsmt.Exposure)), labels = c(levels(Bsmt.Exposure), "NoBa"), exclude = NULL)) %>%
   mutate(BsmtFin.Type.1=  factor(BsmtFin.Type.1, levels = levels(addNA(BsmtFin.Type.1)), labels = c(levels(BsmtFin.Type.1), "NoBa"), exclude = NULL)) %>%
    mutate(BsmtFin.Type.2=  factor(BsmtFin.Type.2, levels = levels(addNA(BsmtFin.Type.2)), labels = c(levels(BsmtFin.Type.2), "NoBa"), exclude = NULL)) %>%
  mutate(X12.SF= X1st.Flr.SF+ X2nd.Flr.SF)  %>%   
  filter(!is.na(Bsmt.Full.Bath)) %>%
  filter(!is.na(Bsmt.Half.Bath)) %>%
  mutate(Baths = Bsmt.Full.Bath + 0.5*Bsmt.Half.Bath + Full.Bath + 0.5*Half.Bath) %>%
  mutate(Fireplace.YN = Fireplaces>0) %>%
  mutate(Fireplace.Qu =  factor(Fireplace.Qu, levels = levels(addNA(Fireplace.Qu)), labels = c(levels(Fireplace.Qu), "None"), exclude = NULL)) %>%
  mutate(Garage.YN = !is.na(Garage.Cond)) %>%
    mutate(Garage.Type =  factor(Garage.Type, levels = levels(addNA(Garage.Type)), labels = c(levels(Garage.Type), "None"), exclude = NULL)) %>%
  mutate(Garage.Finish =  factor(Garage.Finish, levels = levels(addNA(Garage.Finish)), labels = c(levels(Garage.Finish), "None"), exclude = NULL)) %>%
    mutate(Garage.Qual =  factor(Garage.Qual, levels = levels(addNA(Garage.Qual)), labels = c(levels(Garage.Qual), "None"), exclude = NULL)) %>%
      mutate(Garage.Cond =  factor(Garage.Cond, levels = levels(addNA(Garage.Cond)), labels = c(levels(Garage.Cond), "None"), exclude = NULL)) %>%
  mutate(Porch.Area = Wood.Deck.SF+ Open.Porch.SF+Enclosed.Porch+X3Ssn.Porch + Screen.Porch) %>%
  mutate(Pool.YN = Pool.Area>0) %>%
  mutate(Pool.QC =  factor(Pool.QC, levels = levels(addNA(Pool.QC)), labels = c(levels(Pool.QC), "None"), exclude = NULL)) %>%
    mutate(Fence =  factor(Fence, levels = levels(addNA(Fence)), labels = c(levels(Fence), "None"), exclude = NULL)) %>%
  mutate(Misc.Feature =  factor(Misc.Feature, levels = levels(addNA(Misc.Feature)), labels = c(levels(Misc.Feature), "None"), exclude = NULL)) %>%
  mutate(Mo.Sold = as.factor(Mo.Sold)) %>%
    mutate(Yr.Sold = as.factor(Yr.Sold)) %>%
  dplyr::select(-Garage.Yr.Blt) %>% filter(Condition.1 != "RRNe") %>% filter(Kitchen.Qual != "Po")
                  

You should save your predictions in a dataframe with columns for PID (property identifier), fit, predicted values on the test data, and where possible lwr and upr, lower and upper 95% interval estimates for predicting price.

Y = test$price
Yhat = exp(Yhat)

#Bias
mean(Yhat[,1] - Y)

#Maximum Deviation
max(abs(Y - Yhat[,1]))

#Mean Absolute Deviation
mean(abs(Y - Yhat[,1]))

#RMSE
sqrt(mean((Y - Yhat[,1])^2))

#Coverage
mean(Yhat[,"lwr"] < Y & Yhat[,"upr"] > Y)

# name dataframe as predictions! DO NOT CHANGE
predictions = as.data.frame(Yhat)
predictions$PID = test$PID
save(predictions, file="predict.Rdata")

Your models will be evaluated on the following criteria on the test data:

* Bias: Average (Yhat-Y) positive values indicate the model tends to overestimate price (on average) while negative values indicate the model tends to underestimate price.

* Maximum Deviation: Max |Y-Yhat| - identifies the worst prediction made in the validation data set.

* Mean Absolute Deviation: Average |Y-Yhat| - the average error (regardless of sign).

  • Root Mean Square Error: Sqrt Average (Y-Yhat)^2

  • Coverage: Average( lwr < Y < upr)

In order to have a passing wercker badge, your file for predictions needs to be the same length as the test data, with three columns: fitted values, lower CI and upper CI values in that order with names, fit, lwr, and upr respectively.

You will be able to see your scores on the score board (coming soon!). They will be initialized by a predction based on the mean in the training data.

Model Check - Test your prediction on the first observation in the training and test data set to make sure that the model gives a reasonable answer and include this in a supplement of your report. This should be done BY HAND using a calculator (this means use the raw data from the original dataset and manually calculate all transformations and interactions with your calculator)! Models that do not give reasonable answers will be given a minimum 2 letter grade reduction. Also be careful as you cannot use certain transformations [log or inverse x] if a variable has values of 0.

Part II: Complex Model

In this part you may go all out for constructing a best fitting model for predicting housing prices using methods that we have covered this semester. You should feel free to to create any new variables (such as quadratic, interaction, or indicator variables, splines, etc). The variable TotalSq = X1st.Flr.SF+X2nd.Flr.SF was added to the dataframe (that does not include basement area, so you may improve on this. A relative grade is assigned by comparing your fit on the test set to that of your fellow students with bonus points awarded to those who substantially exceed their fellow students and point reductions occurring for models which fit exceedingly poorly.

library(glmnet)
#Design matrices
X.train = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
    Overall.Qual + Overall.Cond + HouseAge + Foundation + Total.Bsmt.SF + 
    Central.Air + log(X12.SF) + Baths + Kitchen.AbvGr + Kitchen.Qual + 
    Functional + Fireplaces + Garage.Cars + Paved.Drive + Bsmt.YN:Bsmt.Exposure, 
    data = data)

X.test = model.matrix(log(price) ~ log(Lot.Area) + Neighborhood + Condition.1 + 
    Overall.Qual + Overall.Cond + HouseAge + Foundation + Total.Bsmt.SF + 
    Central.Air + log(X12.SF) + Baths + Kitchen.AbvGr + Kitchen.Qual + 
    Functional + Fireplaces + Garage.Cars + Paved.Drive + Bsmt.YN:Bsmt.Exposure, 
    data = test)

#Fit lasso regression
price.lasso = glmnet(X.train, log(data$price), alpha=1)

cv.lasso = cv.glmnet(X.train, log(data$price), alpha=1)

#Obtain prediction on test data
yhat.lasso.test = predict(app.lasso, s=cv.lasso$lambda.min, type="response", newx = X.test)

#Compute RMSE for test data
rmse.lasso.test = rmse(test$price, exp(yhat.lasso.test))
rmse.tab$lasso<-rmse.lasso.test

Update your predictions using your complex model to provide point estimates and CI.

You may iterate here as much as you like exploring different models until you are satisfied with your results.

Bayesian lasso

coef(housing.lasso)
1170 x 1 sparse Matrix of class "dgCMatrix"
                                                 s0
(Intercept)                           -2.481733e+05
(Intercept)                            .           
log(Lot.Area)                         -2.820674e+03
NeighborhoodBlueste                    .           
NeighborhoodBrDale                    -2.071429e+04
NeighborhoodBrkSide                   -1.922483e+04
NeighborhoodClearCr                    4.098550e+04
NeighborhoodCollgCr                   -2.140941e+03
NeighborhoodCrawfor                   -2.194616e+04
NeighborhoodEdwards                   -3.741877e+04
NeighborhoodGilbert                   -2.063168e+04
NeighborhoodGreens                    -1.468658e+04
NeighborhoodGrnHill                    7.989575e+04
NeighborhoodIDOTRR                    -3.370340e+04
NeighborhoodLandmrk                    .           
NeighborhoodMeadowV                   -7.435215e+04
NeighborhoodMitchel                    7.567808e+02
NeighborhoodNAmes                      2.532327e+04
NeighborhoodNoRidge                    5.960950e+04
NeighborhoodNPkVill                    .           
NeighborhoodNridgHt                    6.494221e+04
NeighborhoodNWAmes                     .           
NeighborhoodOldTown                   -1.200522e+04
NeighborhoodSawyer                    -2.410915e+04
NeighborhoodSawyerW                   -3.278347e+04
NeighborhoodSomerst                    1.555067e+04
NeighborhoodStoneBr                    3.757409e+04
NeighborhoodSWISU                      7.946636e+03
NeighborhoodTimber                     4.623122e+03
NeighborhoodVeenker                    .           
Condition.1Artery                     -5.160022e+03
Condition.1Feedr                       7.435678e+03
Condition.1Park                        6.811294e+03
Condition.1Rail                       -2.466063e+04
Overall.Qual                          -8.687108e+03
Overall.Cond                          -1.933976e+04
HouseAge                              -5.417731e+02
FoundationCBlock                       3.181746e+02
FoundationPConc                        6.481395e+03
FoundationSlab                        -2.794967e+04
FoundationStone                       -9.737201e+00
FoundationWood                         .           
Total.Bsmt.SF                          1.086050e+01
Central.AirY                          -4.153187e+04
log(X12.SF)                            4.482594e+04
Baths                                 -3.003026e+03
Kitchen.AbvGr                          9.251757e+02
Kitchen.Qual                           6.304757e+02
FunctionalMaj2                        -3.203463e+04
FunctionalMin1                         7.379779e+04
FunctionalMin2                         7.673434e+04
FunctionalMod                         -6.627107e+03
FunctionalTyp                          2.093706e+04
Fireplaces                             .           
Garage.Cars                           -9.906074e+03
Paved.DriveP                          -8.263301e+03
Paved.DriveY                           1.011677e+04
log(Lot.Area):NeighborhoodBlueste      1.969465e+03
log(Lot.Area):NeighborhoodBrDale      -1.439665e+03
log(Lot.Area):NeighborhoodBrkSide      5.167852e+03
log(Lot.Area):NeighborhoodClearCr      1.487235e+00
log(Lot.Area):NeighborhoodCollgCr     -1.866509e+03
log(Lot.Area):NeighborhoodCrawfor      1.481544e+02
log(Lot.Area):NeighborhoodEdwards      2.345952e+03
log(Lot.Area):NeighborhoodGilbert     -4.651634e+03
log(Lot.Area):NeighborhoodGreens       .           
log(Lot.Area):NeighborhoodGrnHill      8.574359e+02
log(Lot.Area):NeighborhoodIDOTRR       1.017452e+03
log(Lot.Area):NeighborhoodLandmrk     -1.469301e+03
log(Lot.Area):NeighborhoodMeadowV     -4.785444e+03
log(Lot.Area):NeighborhoodMitchel     -2.561962e+03
log(Lot.Area):NeighborhoodNAmes       -1.370296e+03
log(Lot.Area):NeighborhoodNoRidge      3.140092e+04
log(Lot.Area):NeighborhoodNPkVill      1.722498e+03
log(Lot.Area):NeighborhoodNridgHt     -4.336223e+03
log(Lot.Area):NeighborhoodNWAmes       1.006834e+01
log(Lot.Area):NeighborhoodOldTown     -4.135321e+03
log(Lot.Area):NeighborhoodSawyer      -6.372042e+03
log(Lot.Area):NeighborhoodSawyerW     -4.773142e+02
log(Lot.Area):NeighborhoodSomerst      5.610437e+03
log(Lot.Area):NeighborhoodStoneBr      5.260326e+03
log(Lot.Area):NeighborhoodSWISU        2.377089e+02
log(Lot.Area):NeighborhoodTimber       7.785457e+03
log(Lot.Area):NeighborhoodVeenker      5.449607e+03
log(Lot.Area):Condition.1Artery        1.906080e+03
log(Lot.Area):Condition.1Feedr        -9.609245e+02
log(Lot.Area):Condition.1Park          2.508378e+03
log(Lot.Area):Condition.1Rail          .           
log(Lot.Area):Overall.Qual             9.696353e+02
log(Lot.Area):Overall.Cond             3.072593e+02
log(Lot.Area):HouseAge                 2.060536e+01
log(Lot.Area):FoundationCBlock         1.799227e+03
log(Lot.Area):FoundationPConc          1.972279e+02
log(Lot.Area):FoundationSlab          -4.428848e+03
log(Lot.Area):FoundationStone          4.257674e+02
log(Lot.Area):FoundationWood           .           
log(Lot.Area):Total.Bsmt.SF            3.608480e+00
log(Lot.Area):Central.AirY            -6.891702e+03
log(Lot.Area):log(X12.SF)              2.298431e+03
log(Lot.Area):Baths                    7.714103e+02
log(Lot.Area):Kitchen.AbvGr           -2.928009e+03
log(Lot.Area):Kitchen.Qual             3.625858e+02
log(Lot.Area):FunctionalMaj2           .           
log(Lot.Area):FunctionalMin1           2.939519e+03
log(Lot.Area):FunctionalMin2           3.815929e+03
log(Lot.Area):FunctionalMod           -9.929820e-01
log(Lot.Area):FunctionalTyp           -5.721209e+02
log(Lot.Area):Fireplaces               .           
log(Lot.Area):Garage.Cars             -4.124218e+02
log(Lot.Area):Paved.DriveP            -5.198722e+03
log(Lot.Area):Paved.DriveY            -3.443184e+03
NeighborhoodBlueste:Condition.1Artery  .           
NeighborhoodBrDale:Condition.1Artery   .           
NeighborhoodBrkSide:Condition.1Artery  .           
NeighborhoodClearCr:Condition.1Artery  .           
NeighborhoodCollgCr:Condition.1Artery  .           
NeighborhoodCrawfor:Condition.1Artery  .           
NeighborhoodEdwards:Condition.1Artery  1.865301e+04
NeighborhoodGilbert:Condition.1Artery  .           
NeighborhoodGreens:Condition.1Artery   .           
NeighborhoodGrnHill:Condition.1Artery  .           
NeighborhoodIDOTRR:Condition.1Artery  -2.866272e+04
NeighborhoodLandmrk:Condition.1Artery  .           
NeighborhoodMeadowV:Condition.1Artery  .           
NeighborhoodMitchel:Condition.1Artery  .           
NeighborhoodNAmes:Condition.1Artery   -7.176780e+03
NeighborhoodNoRidge:Condition.1Artery  .           
NeighborhoodNPkVill:Condition.1Artery  .           
NeighborhoodNridgHt:Condition.1Artery  .           
NeighborhoodNWAmes:Condition.1Artery   .           
NeighborhoodOldTown:Condition.1Artery  .           
NeighborhoodSawyer:Condition.1Artery   .           
NeighborhoodSawyerW:Condition.1Artery  .           
NeighborhoodSomerst:Condition.1Artery  .           
NeighborhoodStoneBr:Condition.1Artery  .           
NeighborhoodSWISU:Condition.1Artery   -2.692082e+04
NeighborhoodTimber:Condition.1Artery   .           
NeighborhoodVeenker:Condition.1Artery  .           
NeighborhoodBlueste:Condition.1Feedr   .           
NeighborhoodBrDale:Condition.1Feedr    .           
NeighborhoodBrkSide:Condition.1Feedr   1.384133e+04
NeighborhoodClearCr:Condition.1Feedr  -2.708729e+04
NeighborhoodCollgCr:Condition.1Feedr   .           
NeighborhoodCrawfor:Condition.1Feedr   1.660394e+03
NeighborhoodEdwards:Condition.1Feedr   1.066244e+04
NeighborhoodGilbert:Condition.1Feedr  -1.520833e+04
NeighborhoodGreens:Condition.1Feedr    .           
NeighborhoodGrnHill:Condition.1Feedr   .           
NeighborhoodIDOTRR:Condition.1Feedr   -2.489069e+04
NeighborhoodLandmrk:Condition.1Feedr   .           
NeighborhoodMeadowV:Condition.1Feedr   .           
NeighborhoodMitchel:Condition.1Feedr   .           
NeighborhoodNAmes:Condition.1Feedr    -2.201557e+00
NeighborhoodNoRidge:Condition.1Feedr   .           
NeighborhoodNPkVill:Condition.1Feedr   .           
NeighborhoodNridgHt:Condition.1Feedr   .           
NeighborhoodNWAmes:Condition.1Feedr    9.716783e+03
NeighborhoodOldTown:Condition.1Feedr   1.707263e+04
NeighborhoodSawyer:Condition.1Feedr    4.306266e+03
NeighborhoodSawyerW:Condition.1Feedr   2.616033e+04
NeighborhoodSomerst:Condition.1Feedr   1.167382e+04
NeighborhoodStoneBr:Condition.1Feedr   .           
NeighborhoodSWISU:Condition.1Feedr     2.686645e+04
NeighborhoodTimber:Condition.1Feedr    .           
NeighborhoodVeenker:Condition.1Feedr   3.782459e+04
NeighborhoodBlueste:Condition.1Park    .           
NeighborhoodBrDale:Condition.1Park     .           
NeighborhoodBrkSide:Condition.1Park    .           
NeighborhoodClearCr:Condition.1Park    .           
NeighborhoodCollgCr:Condition.1Park   -5.223636e+03
NeighborhoodCrawfor:Condition.1Park    .           
NeighborhoodEdwards:Condition.1Park    .           
NeighborhoodGilbert:Condition.1Park    .           
NeighborhoodGreens:Condition.1Park     .           
NeighborhoodGrnHill:Condition.1Park    .           
NeighborhoodIDOTRR:Condition.1Park     .           
NeighborhoodLandmrk:Condition.1Park    .           
NeighborhoodMeadowV:Condition.1Park    .           
NeighborhoodMitchel:Condition.1Park    .           
NeighborhoodNAmes:Condition.1Park     -8.382130e+02
NeighborhoodNoRidge:Condition.1Park    .           
NeighborhoodNPkVill:Condition.1Park    .           
NeighborhoodNridgHt:Condition.1Park   -8.323452e+02
NeighborhoodNWAmes:Condition.1Park     .           
NeighborhoodOldTown:Condition.1Park    .           
NeighborhoodSawyer:Condition.1Park     .           
NeighborhoodSawyerW:Condition.1Park    .           
NeighborhoodSomerst:Condition.1Park   -9.615583e+03
NeighborhoodStoneBr:Condition.1Park   -2.173948e+04
NeighborhoodSWISU:Condition.1Park      .           
NeighborhoodTimber:Condition.1Park     .           
NeighborhoodVeenker:Condition.1Park    .           
NeighborhoodBlueste:Condition.1Rail    .           
NeighborhoodBrDale:Condition.1Rail     .           
NeighborhoodBrkSide:Condition.1Rail    1.579995e+04
NeighborhoodClearCr:Condition.1Rail    .           
NeighborhoodCollgCr:Condition.1Rail    .           
NeighborhoodCrawfor:Condition.1Rail    .           
NeighborhoodEdwards:Condition.1Rail    .           
NeighborhoodGilbert:Condition.1Rail   -3.785934e+04
NeighborhoodGreens:Condition.1Rail     .           
NeighborhoodGrnHill:Condition.1Rail    .           
NeighborhoodIDOTRR:Condition.1Rail     2.212995e+04
NeighborhoodLandmrk:Condition.1Rail    .           
NeighborhoodMeadowV:Condition.1Rail    .           
NeighborhoodMitchel:Condition.1Rail    .           
NeighborhoodNAmes:Condition.1Rail      .           
NeighborhoodNoRidge:Condition.1Rail    .           
NeighborhoodNPkVill:Condition.1Rail    .           
NeighborhoodNridgHt:Condition.1Rail    .           
NeighborhoodNWAmes:Condition.1Rail     1.051407e+04
NeighborhoodOldTown:Condition.1Rail    4.317082e+04
NeighborhoodSawyer:Condition.1Rail     6.821910e+03
NeighborhoodSawyerW:Condition.1Rail   -2.095610e+04
NeighborhoodSomerst:Condition.1Rail    2.077964e+03
NeighborhoodStoneBr:Condition.1Rail    .           
NeighborhoodSWISU:Condition.1Rail      .           
NeighborhoodTimber:Condition.1Rail     .           
NeighborhoodVeenker:Condition.1Rail    .           
NeighborhoodBlueste:Overall.Qual       1.278702e+04
NeighborhoodBrDale:Overall.Qual        1.280317e+03
NeighborhoodBrkSide:Overall.Qual       4.509090e+02
NeighborhoodClearCr:Overall.Qual       9.120932e+03
NeighborhoodCollgCr:Overall.Qual      -2.887457e+03
NeighborhoodCrawfor:Overall.Qual       2.288150e+03
NeighborhoodEdwards:Overall.Qual      -7.863646e+03
NeighborhoodGilbert:Overall.Qual       3.356545e+03
NeighborhoodGreens:Overall.Qual        1.387156e+03
NeighborhoodGrnHill:Overall.Qual       1.129114e+03
NeighborhoodIDOTRR:Overall.Qual       -2.210951e+03
NeighborhoodLandmrk:Overall.Qual       .           
NeighborhoodMeadowV:Overall.Qual      -6.782732e+03
NeighborhoodMitchel:Overall.Qual      -6.464510e+03
NeighborhoodNAmes:Overall.Qual        -2.966498e+03
NeighborhoodNoRidge:Overall.Qual      -2.383321e+03
NeighborhoodNPkVill:Overall.Qual      -1.126953e+04
NeighborhoodNridgHt:Overall.Qual       6.547708e+02
NeighborhoodNWAmes:Overall.Qual       -5.969528e+03
NeighborhoodOldTown:Overall.Qual      -1.789256e+03
NeighborhoodSawyer:Overall.Qual       -4.263807e+03
NeighborhoodSawyerW:Overall.Qual      -2.596599e+03
NeighborhoodSomerst:Overall.Qual      -8.919578e+02
NeighborhoodStoneBr:Overall.Qual      -1.173775e+04
NeighborhoodSWISU:Overall.Qual        -5.900090e+02
NeighborhoodTimber:Overall.Qual        .           
NeighborhoodVeenker:Overall.Qual      -1.926608e+03
NeighborhoodBlueste:Overall.Cond       5.800491e+03
NeighborhoodBrDale:Overall.Cond        4.294752e+03
NeighborhoodBrkSide:Overall.Cond      -6.818636e+02
NeighborhoodClearCr:Overall.Cond       3.576969e+03
NeighborhoodCollgCr:Overall.Cond       6.087199e+03
NeighborhoodCrawfor:Overall.Cond      -4.536931e+03
NeighborhoodEdwards:Overall.Cond       8.003291e+02
NeighborhoodGilbert:Overall.Cond       6.970022e+03
NeighborhoodGreens:Overall.Cond        8.502759e+03
NeighborhoodGrnHill:Overall.Cond       .           
NeighborhoodIDOTRR:Overall.Cond        3.014450e+03
NeighborhoodLandmrk:Overall.Cond       .           
NeighborhoodMeadowV:Overall.Cond      -1.986557e+03
NeighborhoodMitchel:Overall.Cond      -1.014271e+03
NeighborhoodNAmes:Overall.Cond        -2.895080e+02
NeighborhoodNoRidge:Overall.Cond      -1.245017e+04
NeighborhoodNPkVill:Overall.Cond       5.098328e+03
NeighborhoodNridgHt:Overall.Cond      -1.884113e+03
NeighborhoodNWAmes:Overall.Cond        3.578541e+03
NeighborhoodOldTown:Overall.Cond       7.459405e+02
NeighborhoodSawyer:Overall.Cond        3.681778e+03
NeighborhoodSawyerW:Overall.Cond       5.527394e+03
NeighborhoodSomerst:Overall.Cond      -6.758819e+03
NeighborhoodStoneBr:Overall.Cond       2.448421e+02
NeighborhoodSWISU:Overall.Cond        -6.103562e+03
NeighborhoodTimber:Overall.Cond       -7.422249e+03
NeighborhoodVeenker:Overall.Cond      -1.897816e+04
NeighborhoodBlueste:HouseAge          -1.585781e+02
NeighborhoodBrDale:HouseAge            2.817410e+02
NeighborhoodBrkSide:HouseAge           2.616997e+02
NeighborhoodClearCr:HouseAge           6.843583e+02
NeighborhoodCollgCr:HouseAge           4.756420e+02
NeighborhoodCrawfor:HouseAge           1.009226e+00
NeighborhoodEdwards:HouseAge          -3.853751e+01
NeighborhoodGilbert:HouseAge          -8.908637e+01
NeighborhoodGreens:HouseAge            9.156589e+02
NeighborhoodGrnHill:HouseAge           1.861608e+02
NeighborhoodIDOTRR:HouseAge            3.298973e+02
NeighborhoodLandmrk:HouseAge           .           
NeighborhoodMeadowV:HouseAge           1.151213e+02
NeighborhoodMitchel:HouseAge           5.838366e+01
NeighborhoodNAmes:HouseAge             1.942339e+02
NeighborhoodNoRidge:HouseAge          -6.236985e+02
NeighborhoodNPkVill:HouseAge           3.952497e+02
NeighborhoodNridgHt:HouseAge          -1.192641e+03
NeighborhoodNWAmes:HouseAge           -5.735172e+00
NeighborhoodOldTown:HouseAge           2.883042e+02
NeighborhoodSawyer:HouseAge            3.830832e+02
NeighborhoodSawyerW:HouseAge           2.092397e+02
NeighborhoodSomerst:HouseAge           1.433830e+03
NeighborhoodStoneBr:HouseAge          -1.529292e+03
NeighborhoodSWISU:HouseAge            -4.789039e+01
NeighborhoodTimber:HouseAge           -2.824966e+02
NeighborhoodVeenker:HouseAge          -1.591223e+03
NeighborhoodBlueste:FoundationCBlock  -7.670598e+04
NeighborhoodBrDale:FoundationCBlock    1.194988e+03
NeighborhoodBrkSide:FoundationCBlock   1.312685e+04
NeighborhoodClearCr:FoundationCBlock  -1.095257e+05
NeighborhoodCollgCr:FoundationCBlock  -1.305436e+04
NeighborhoodCrawfor:FoundationCBlock   3.376332e+04
NeighborhoodEdwards:FoundationCBlock   2.799478e+04
NeighborhoodGilbert:FoundationCBlock  -4.657422e+04
NeighborhoodGreens:FoundationCBlock   -3.348628e+04
NeighborhoodGrnHill:FoundationCBlock   3.321382e+03
NeighborhoodIDOTRR:FoundationCBlock    1.742947e+04
NeighborhoodLandmrk:FoundationCBlock   .           
NeighborhoodMeadowV:FoundationCBlock   9.142792e+04
NeighborhoodMitchel:FoundationCBlock   6.711466e+04
NeighborhoodNAmes:FoundationCBlock     1.799054e+04
NeighborhoodNoRidge:FoundationCBlock   .           
NeighborhoodNPkVill:FoundationCBlock   1.880958e+04
NeighborhoodNridgHt:FoundationCBlock   .           
NeighborhoodNWAmes:FoundationCBlock    3.850965e+03
NeighborhoodOldTown:FoundationCBlock   1.375454e+04
NeighborhoodSawyer:FoundationCBlock    1.177385e+04
NeighborhoodSawyerW:FoundationCBlock   1.285960e+03
NeighborhoodSomerst:FoundationCBlock   .           
NeighborhoodStoneBr:FoundationCBlock   3.484069e+04
NeighborhoodSWISU:FoundationCBlock     5.202907e+03
NeighborhoodTimber:FoundationCBlock   -5.418507e+04
NeighborhoodVeenker:FoundationCBlock   4.324409e+04
NeighborhoodBlueste:FoundationPConc    .           
NeighborhoodBrDale:FoundationPConc     .           
NeighborhoodBrkSide:FoundationPConc    1.555431e+04
NeighborhoodClearCr:FoundationPConc   -7.458508e+04
NeighborhoodCollgCr:FoundationPConc    .           
NeighborhoodCrawfor:FoundationPConc    3.840414e+04
NeighborhoodEdwards:FoundationPConc    3.017124e+04
NeighborhoodGilbert:FoundationPConc    2.678701e+03
NeighborhoodGreens:FoundationPConc     .           
NeighborhoodGrnHill:FoundationPConc    .           
NeighborhoodIDOTRR:FoundationPConc     2.683037e+04
NeighborhoodLandmrk:FoundationPConc    .           
NeighborhoodMeadowV:FoundationPConc    9.974146e+04
NeighborhoodMitchel:FoundationPConc    6.000149e+04
NeighborhoodNAmes:FoundationPConc      2.005812e+04
NeighborhoodNoRidge:FoundationPConc   -2.399086e+05
NeighborhoodNPkVill:FoundationPConc    .           
NeighborhoodNridgHt:FoundationPConc    .           
NeighborhoodNWAmes:FoundationPConc     1.183912e+04
NeighborhoodOldTown:FoundationPConc    1.507170e+04
NeighborhoodSawyer:FoundationPConc     1.339285e+04
NeighborhoodSawyerW:FoundationPConc   -1.419901e+03
NeighborhoodSomerst:FoundationPConc   -3.273427e+03
NeighborhoodStoneBr:FoundationPConc    6.186384e+03
NeighborhoodSWISU:FoundationPConc      2.033934e+04
NeighborhoodTimber:FoundationPConc    -3.695573e+04
NeighborhoodVeenker:FoundationPConc    3.070993e+04
NeighborhoodBlueste:FoundationSlab     .           
NeighborhoodBrDale:FoundationSlab      .           
NeighborhoodBrkSide:FoundationSlab     2.761503e+04
NeighborhoodClearCr:FoundationSlab     .           
NeighborhoodCollgCr:FoundationSlab     .           
NeighborhoodCrawfor:FoundationSlab     .           
NeighborhoodEdwards:FoundationSlab     5.309315e+04
NeighborhoodGilbert:FoundationSlab     .           
NeighborhoodGreens:FoundationSlab      .           
NeighborhoodGrnHill:FoundationSlab     .           
NeighborhoodIDOTRR:FoundationSlab      3.810880e+04
NeighborhoodLandmrk:FoundationSlab     .           
NeighborhoodMeadowV:FoundationSlab     .           
NeighborhoodMitchel:FoundationSlab     5.955324e+04
NeighborhoodNAmes:FoundationSlab       1.810422e+04
NeighborhoodNoRidge:FoundationSlab     .           
NeighborhoodNPkVill:FoundationSlab     .           
NeighborhoodNridgHt:FoundationSlab     .           
NeighborhoodNWAmes:FoundationSlab      .           
NeighborhoodOldTown:FoundationSlab     1.051785e+04
NeighborhoodSawyer:FoundationSlab     -1.883762e+04
NeighborhoodSawyerW:FoundationSlab     6.456851e+03
NeighborhoodSomerst:FoundationSlab     .           
NeighborhoodStoneBr:FoundationSlab     .           
NeighborhoodSWISU:FoundationSlab       1.007064e+05
NeighborhoodTimber:FoundationSlab      .           
NeighborhoodVeenker:FoundationSlab     .           
NeighborhoodBlueste:FoundationStone    .           
NeighborhoodBrDale:FoundationStone     .           
NeighborhoodBrkSide:FoundationStone    .           
NeighborhoodClearCr:FoundationStone    .           
NeighborhoodCollgCr:FoundationStone    .           
NeighborhoodCrawfor:FoundationStone    1.418961e+04
NeighborhoodEdwards:FoundationStone    .           
NeighborhoodGilbert:FoundationStone    .           
NeighborhoodGreens:FoundationStone     .           
NeighborhoodGrnHill:FoundationStone    .           
NeighborhoodIDOTRR:FoundationStone    -8.295564e+04
NeighborhoodLandmrk:FoundationStone    .           
NeighborhoodMeadowV:FoundationStone    .           
NeighborhoodMitchel:FoundationStone    .           
NeighborhoodNAmes:FoundationStone      3.948740e+03
NeighborhoodNoRidge:FoundationStone    .           
NeighborhoodNPkVill:FoundationStone    .           
NeighborhoodNridgHt:FoundationStone    .           
NeighborhoodNWAmes:FoundationStone     .           
NeighborhoodOldTown:FoundationStone    .           
NeighborhoodSawyer:FoundationStone     .           
NeighborhoodSawyerW:FoundationStone    .           
NeighborhoodSomerst:FoundationStone    .           
NeighborhoodStoneBr:FoundationStone    .           
NeighborhoodSWISU:FoundationStone      .           
NeighborhoodTimber:FoundationStone     .           
NeighborhoodVeenker:FoundationStone    .           
NeighborhoodBlueste:FoundationWood     .           
NeighborhoodBrDale:FoundationWood      .           
NeighborhoodBrkSide:FoundationWood     .           
NeighborhoodClearCr:FoundationWood     .           
NeighborhoodCollgCr:FoundationWood     .           
NeighborhoodCrawfor:FoundationWood     .           
NeighborhoodEdwards:FoundationWood     .           
NeighborhoodGilbert:FoundationWood    -8.825003e+03
NeighborhoodGreens:FoundationWood      .           
NeighborhoodGrnHill:FoundationWood     .           
NeighborhoodIDOTRR:FoundationWood      .           
NeighborhoodLandmrk:FoundationWood     .           
NeighborhoodMeadowV:FoundationWood     .           
NeighborhoodMitchel:FoundationWood     .           
NeighborhoodNAmes:FoundationWood       .           
NeighborhoodNoRidge:FoundationWood     .           
NeighborhoodNPkVill:FoundationWood     .           
NeighborhoodNridgHt:FoundationWood     .           
NeighborhoodNWAmes:FoundationWood      .           
NeighborhoodOldTown:FoundationWood     .           
NeighborhoodSawyer:FoundationWood      .           
NeighborhoodSawyerW:FoundationWood     .           
NeighborhoodSomerst:FoundationWood     .           
NeighborhoodStoneBr:FoundationWood     .           
NeighborhoodSWISU:FoundationWood       .           
NeighborhoodTimber:FoundationWood      .           
NeighborhoodVeenker:FoundationWood     .           
NeighborhoodBlueste:Total.Bsmt.SF     -6.195864e+01
NeighborhoodBrDale:Total.Bsmt.SF      -3.025655e+01
NeighborhoodBrkSide:Total.Bsmt.SF      9.448265e-01
NeighborhoodClearCr:Total.Bsmt.SF      1.458647e+01
NeighborhoodCollgCr:Total.Bsmt.SF     -7.984066e+00
NeighborhoodCrawfor:Total.Bsmt.SF     -4.388974e+01
NeighborhoodEdwards:Total.Bsmt.SF     -2.795649e-02
NeighborhoodGilbert:Total.Bsmt.SF      .           
NeighborhoodGreens:Total.Bsmt.SF      -3.501130e+01
NeighborhoodGrnHill:Total.Bsmt.SF      .           
NeighborhoodIDOTRR:Total.Bsmt.SF      -1.841197e+01
NeighborhoodLandmrk:Total.Bsmt.SF      .           
NeighborhoodMeadowV:Total.Bsmt.SF     -1.120302e+00
NeighborhoodMitchel:Total.Bsmt.SF      1.876016e+01
NeighborhoodNAmes:Total.Bsmt.SF       -1.316175e+01
NeighborhoodNoRidge:Total.Bsmt.SF     -7.402320e+00
NeighborhoodNPkVill:Total.Bsmt.SF      1.540046e+01
NeighborhoodNridgHt:Total.Bsmt.SF      8.899627e+00
NeighborhoodNWAmes:Total.Bsmt.SF      -1.560533e+01
NeighborhoodOldTown:Total.Bsmt.SF     -1.072386e+01
NeighborhoodSawyer:Total.Bsmt.SF      -5.284939e+00
NeighborhoodSawyerW:Total.Bsmt.SF     -9.205049e+00
NeighborhoodSomerst:Total.Bsmt.SF      1.697050e+00
NeighborhoodStoneBr:Total.Bsmt.SF      6.252423e+01
NeighborhoodSWISU:Total.Bsmt.SF       -5.135103e+01
NeighborhoodTimber:Total.Bsmt.SF       1.482535e-02
NeighborhoodVeenker:Total.Bsmt.SF      1.402837e+01
NeighborhoodBlueste:Central.AirY       .           
NeighborhoodBrDale:Central.AirY        .           
NeighborhoodBrkSide:Central.AirY       1.326769e+04
NeighborhoodClearCr:Central.AirY      -2.613468e+04
NeighborhoodCollgCr:Central.AirY       2.291435e+03
NeighborhoodCrawfor:Central.AirY       3.228596e+04
NeighborhoodEdwards:Central.AirY       2.076208e+04
NeighborhoodGilbert:Central.AirY      -7.812483e+03
NeighborhoodGreens:Central.AirY        .           
NeighborhoodGrnHill:Central.AirY       3.542000e+02
NeighborhoodIDOTRR:Central.AirY        3.945154e+04
NeighborhoodLandmrk:Central.AirY       .           
NeighborhoodMeadowV:Central.AirY       8.393022e+03
NeighborhoodMitchel:Central.AirY      -2.922281e+03
NeighborhoodNAmes:Central.AirY        -1.189453e+04
NeighborhoodNoRidge:Central.AirY       .           
NeighborhoodNPkVill:Central.AirY       .           
NeighborhoodNridgHt:Central.AirY      -1.065700e+04
NeighborhoodNWAmes:Central.AirY        3.715909e+03
NeighborhoodOldTown:Central.AirY       2.967330e+04
NeighborhoodSawyer:Central.AirY        2.615836e+04
NeighborhoodSawyerW:Central.AirY      -1.775825e+03
NeighborhoodSomerst:Central.AirY      -3.837463e+03
NeighborhoodStoneBr:Central.AirY      -6.277786e+04
NeighborhoodSWISU:Central.AirY         2.944683e+04
NeighborhoodTimber:Central.AirY       -6.722770e+03
NeighborhoodVeenker:Central.AirY       1.648501e+03
NeighborhoodBlueste:log(X12.SF)        .           
NeighborhoodBrDale:log(X12.SF)         3.896258e+02
NeighborhoodBrkSide:log(X12.SF)       -4.289714e+03
NeighborhoodClearCr:log(X12.SF)       -8.923654e+03
NeighborhoodCollgCr:log(X12.SF)        6.495664e+02
NeighborhoodCrawfor:log(X12.SF)        1.438919e+04
NeighborhoodEdwards:log(X12.SF)       -1.357257e+04
NeighborhoodGilbert:log(X12.SF)       -1.225397e+02
NeighborhoodGreens:log(X12.SF)        -3.341844e+02
NeighborhoodGrnHill:log(X12.SF)        .           
NeighborhoodIDOTRR:log(X12.SF)        -1.735208e+04

 ..............................
 ........suppressing rows in show(); maybe adjust 'options(max.print= *, width = *)'
 ..............................

                                              s0
NeighborhoodOldTown:FunctionalMin2  6.052447e+04
NeighborhoodSawyer:FunctionalMin2   4.161795e+04
NeighborhoodSawyerW:FunctionalMin2 -7.150591e+03
NeighborhoodSomerst:FunctionalMin2 -3.311685e+04
NeighborhoodStoneBr:FunctionalMin2  .           
NeighborhoodSWISU:FunctionalMin2    2.798351e+04
NeighborhoodTimber:FunctionalMin2   .           
NeighborhoodVeenker:FunctionalMin2  .           
NeighborhoodBlueste:FunctionalMod   .           
NeighborhoodBrDale:FunctionalMod    .           
NeighborhoodBrkSide:FunctionalMod  -4.555339e+04
NeighborhoodClearCr:FunctionalMod  -5.515845e+04
NeighborhoodCollgCr:FunctionalMod   .           
NeighborhoodCrawfor:FunctionalMod   .           
NeighborhoodEdwards:FunctionalMod   2.113144e+04
NeighborhoodGilbert:FunctionalMod   .           
NeighborhoodGreens:FunctionalMod    .           
NeighborhoodGrnHill:FunctionalMod   .           
NeighborhoodIDOTRR:FunctionalMod    1.539571e+04
NeighborhoodLandmrk:FunctionalMod   .           
NeighborhoodMeadowV:FunctionalMod   .           
NeighborhoodMitchel:FunctionalMod   .           
NeighborhoodNAmes:FunctionalMod     2.030644e+02
NeighborhoodNoRidge:FunctionalMod   .           
NeighborhoodNPkVill:FunctionalMod   .           
NeighborhoodNridgHt:FunctionalMod   .           
NeighborhoodNWAmes:FunctionalMod    .           
NeighborhoodOldTown:FunctionalMod   3.754003e+04
NeighborhoodSawyer:FunctionalMod   -1.365215e+04
NeighborhoodSawyerW:FunctionalMod  -3.950334e+04
NeighborhoodSomerst:FunctionalMod   .           
NeighborhoodStoneBr:FunctionalMod   2.230404e+04
NeighborhoodSWISU:FunctionalMod     .           
NeighborhoodTimber:FunctionalMod    .           
NeighborhoodVeenker:FunctionalMod  -8.205382e+02
NeighborhoodBlueste:FunctionalTyp   8.910605e+02
NeighborhoodBrDale:FunctionalTyp    .           
NeighborhoodBrkSide:FunctionalTyp  -5.907216e+04
NeighborhoodClearCr:FunctionalTyp   1.007998e+04
NeighborhoodCollgCr:FunctionalTyp  -2.905831e+03
NeighborhoodCrawfor:FunctionalTyp   9.723137e+03
NeighborhoodEdwards:FunctionalTyp   1.811535e+04
NeighborhoodGilbert:FunctionalTyp  -1.038670e+02
NeighborhoodGreens:FunctionalTyp    .           
NeighborhoodGrnHill:FunctionalTyp   .           
NeighborhoodIDOTRR:FunctionalTyp    3.608659e+04
NeighborhoodLandmrk:FunctionalTyp   .           
NeighborhoodMeadowV:FunctionalTyp  -2.435448e+04
NeighborhoodMitchel:FunctionalTyp   6.309683e+03
NeighborhoodNAmes:FunctionalTyp    -1.030705e+04
NeighborhoodNoRidge:FunctionalTyp  -6.959313e+04
NeighborhoodNPkVill:FunctionalTyp  -7.885402e+02
NeighborhoodNridgHt:FunctionalTyp  -3.785875e+04
NeighborhoodNWAmes:FunctionalTyp    4.227402e+04
NeighborhoodOldTown:FunctionalTyp   6.134928e+04
NeighborhoodSawyer:FunctionalTyp    4.287953e+04
NeighborhoodSawyerW:FunctionalTyp  -2.096883e+04
NeighborhoodSomerst:FunctionalTyp  -3.785206e+04
NeighborhoodStoneBr:FunctionalTyp  -3.541593e+04
NeighborhoodSWISU:FunctionalTyp     3.466531e+04
NeighborhoodTimber:FunctionalTyp   -3.372127e+04
NeighborhoodVeenker:FunctionalTyp   1.043751e+05
NeighborhoodBlueste:Fireplaces     -4.303970e+04
NeighborhoodBrDale:Fireplaces      -5.406090e+03
NeighborhoodBrkSide:Fireplaces     -8.813769e+03
NeighborhoodClearCr:Fireplaces     -4.054900e+03
NeighborhoodCollgCr:Fireplaces     -2.087977e+03
NeighborhoodCrawfor:Fireplaces      1.177167e+03
NeighborhoodEdwards:Fireplaces      3.554600e+03
NeighborhoodGilbert:Fireplaces     -2.856974e+03
NeighborhoodGreens:Fireplaces       1.284140e+04
NeighborhoodGrnHill:Fireplaces      .           
NeighborhoodIDOTRR:Fireplaces       1.820482e+03
NeighborhoodLandmrk:Fireplaces      .           
NeighborhoodMeadowV:Fireplaces     -1.093472e+04
NeighborhoodMitchel:Fireplaces     -6.468019e+02
NeighborhoodNAmes:Fireplaces       -6.018056e+03
NeighborhoodNoRidge:Fireplaces     -1.151900e+04
NeighborhoodNPkVill:Fireplaces     -4.061900e+04
NeighborhoodNridgHt:Fireplaces      2.193598e+04
NeighborhoodNWAmes:Fireplaces      -1.118831e+04
NeighborhoodOldTown:Fireplaces     -5.539852e+03
NeighborhoodSawyer:Fireplaces      -5.016812e+03
NeighborhoodSawyerW:Fireplaces      1.334180e+03
NeighborhoodSomerst:Fireplaces      8.228491e+03
NeighborhoodStoneBr:Fireplaces      1.737667e+02
NeighborhoodSWISU:Fireplaces       -7.926273e+03
NeighborhoodTimber:Fireplaces       8.246769e+03
NeighborhoodVeenker:Fireplaces     -2.377451e+04
NeighborhoodBlueste:Garage.Cars     1.879425e+04
NeighborhoodBrDale:Garage.Cars      3.878644e+03
NeighborhoodBrkSide:Garage.Cars     2.433543e+04
NeighborhoodClearCr:Garage.Cars     1.913935e+04
NeighborhoodCollgCr:Garage.Cars     2.467441e+04
NeighborhoodCrawfor:Garage.Cars     1.056673e+04
NeighborhoodEdwards:Garage.Cars     1.495691e+04
NeighborhoodGilbert:Garage.Cars     2.958493e+04
NeighborhoodGreens:Garage.Cars      .           
NeighborhoodGrnHill:Garage.Cars     .           
NeighborhoodIDOTRR:Garage.Cars      1.746818e+04
NeighborhoodLandmrk:Garage.Cars     .           
NeighborhoodMeadowV:Garage.Cars     1.933489e+04
NeighborhoodMitchel:Garage.Cars     1.871638e+04
NeighborhoodNAmes:Garage.Cars       1.750737e+04
NeighborhoodNoRidge:Garage.Cars     2.495561e+04
NeighborhoodNPkVill:Garage.Cars     3.349815e+03
NeighborhoodNridgHt:Garage.Cars     3.007006e+04
NeighborhoodNWAmes:Garage.Cars      1.524504e+04
NeighborhoodOldTown:Garage.Cars     1.519914e+04
NeighborhoodSawyer:Garage.Cars      1.579902e+04
NeighborhoodSawyerW:Garage.Cars     9.375270e+03
NeighborhoodSomerst:Garage.Cars     2.958286e+03
NeighborhoodStoneBr:Garage.Cars     4.022816e+04
NeighborhoodSWISU:Garage.Cars       7.792872e+03
NeighborhoodTimber:Garage.Cars      1.840704e+04
NeighborhoodVeenker:Garage.Cars     1.764819e+04
NeighborhoodBlueste:Paved.DriveP    .           
NeighborhoodBrDale:Paved.DriveP     .           
NeighborhoodBrkSide:Paved.DriveP    6.960465e+03
NeighborhoodClearCr:Paved.DriveP    .           
NeighborhoodCollgCr:Paved.DriveP    .           
NeighborhoodCrawfor:Paved.DriveP    7.966641e+03
NeighborhoodEdwards:Paved.DriveP    2.965882e+04
NeighborhoodGilbert:Paved.DriveP    .           
NeighborhoodGreens:Paved.DriveP     .           
NeighborhoodGrnHill:Paved.DriveP    .           
NeighborhoodIDOTRR:Paved.DriveP     6.657340e+02
NeighborhoodLandmrk:Paved.DriveP    .           
NeighborhoodMeadowV:Paved.DriveP    .           
NeighborhoodMitchel:Paved.DriveP    1.064167e+04
NeighborhoodNAmes:Paved.DriveP      .           
NeighborhoodNoRidge:Paved.DriveP    .           
NeighborhoodNPkVill:Paved.DriveP    .           
NeighborhoodNridgHt:Paved.DriveP    .           
NeighborhoodNWAmes:Paved.DriveP     .           
NeighborhoodOldTown:Paved.DriveP    6.159125e+02
NeighborhoodSawyer:Paved.DriveP     .           
NeighborhoodSawyerW:Paved.DriveP    .           
NeighborhoodSomerst:Paved.DriveP    .           
NeighborhoodStoneBr:Paved.DriveP    .           
NeighborhoodSWISU:Paved.DriveP      4.117996e+04
NeighborhoodTimber:Paved.DriveP     .           
NeighborhoodVeenker:Paved.DriveP    .           
NeighborhoodBlueste:Paved.DriveY    3.215277e+04
NeighborhoodBrDale:Paved.DriveY     1.284032e+04
NeighborhoodBrkSide:Paved.DriveY   -1.391058e+03
NeighborhoodClearCr:Paved.DriveY    6.993697e+03
NeighborhoodCollgCr:Paved.DriveY   -1.192901e+04
NeighborhoodCrawfor:Paved.DriveY    2.691864e+04
NeighborhoodEdwards:Paved.DriveY    1.984422e+04
NeighborhoodGilbert:Paved.DriveY   -2.135256e+04
NeighborhoodGreens:Paved.DriveY     .           
NeighborhoodGrnHill:Paved.DriveY    2.355415e+03
NeighborhoodIDOTRR:Paved.DriveY     1.554200e+03
NeighborhoodLandmrk:Paved.DriveY    .           
NeighborhoodMeadowV:Paved.DriveY    .           
NeighborhoodMitchel:Paved.DriveY    6.896190e+01
NeighborhoodNAmes:Paved.DriveY      1.606338e+04
NeighborhoodNoRidge:Paved.DriveY   -1.604663e+04
NeighborhoodNPkVill:Paved.DriveY    4.393436e+04
NeighborhoodNridgHt:Paved.DriveY   -6.663016e+04
NeighborhoodNWAmes:Paved.DriveY    -3.067297e+04
NeighborhoodOldTown:Paved.DriveY    6.233042e+03
NeighborhoodSawyer:Paved.DriveY     4.592254e+03
NeighborhoodSawyerW:Paved.DriveY    4.834718e+04
NeighborhoodSomerst:Paved.DriveY    1.468738e+03
NeighborhoodStoneBr:Paved.DriveY   -5.827632e+04
NeighborhoodSWISU:Paved.DriveY      4.429564e+04
NeighborhoodTimber:Paved.DriveY    -1.705226e+04
NeighborhoodVeenker:Paved.DriveY   -4.510289e+04
Condition.1Artery:Overall.Qual      2.351656e+03
Condition.1Feedr:Overall.Qual      -4.107351e+03
Condition.1Park:Overall.Qual        8.747319e+02
Condition.1Rail:Overall.Qual        1.577419e+03
Condition.1Artery:Overall.Cond      2.298631e+03
Condition.1Feedr:Overall.Cond       1.701441e+03
Condition.1Park:Overall.Cond        3.508284e+03
Condition.1Rail:Overall.Cond       -1.316311e+04
Condition.1Artery:HouseAge          1.719589e+02
Condition.1Feedr:HouseAge          -4.364914e+01
Condition.1Park:HouseAge            4.424944e+01
Condition.1Rail:HouseAge           -7.091646e+02
Condition.1Artery:FoundationCBlock  7.311144e+03
Condition.1Feedr:FoundationCBlock   8.927356e+03
Condition.1Park:FoundationCBlock   -3.511680e+04
Condition.1Rail:FoundationCBlock    1.226001e+04
Condition.1Artery:FoundationPConc   1.165727e+04
Condition.1Feedr:FoundationPConc    3.362105e+03
Condition.1Park:FoundationPConc    -3.218194e+04
Condition.1Rail:FoundationPConc     3.939639e+04
Condition.1Artery:FoundationSlab    .           
Condition.1Feedr:FoundationSlab     2.086550e+04
Condition.1Park:FoundationSlab     -2.610248e+04
Condition.1Rail:FoundationSlab      .           
Condition.1Artery:FoundationStone   .           
Condition.1Feedr:FoundationStone    .           
Condition.1Park:FoundationStone     .           
Condition.1Rail:FoundationStone     .           
Condition.1Artery:FoundationWood    .           
Condition.1Feedr:FoundationWood     .           
Condition.1Park:FoundationWood      .           
Condition.1Rail:FoundationWood      .           
Condition.1Artery:Total.Bsmt.SF     3.067522e+01
Condition.1Feedr:Total.Bsmt.SF      2.278623e+01
Condition.1Park:Total.Bsmt.SF       1.003780e+01
Condition.1Rail:Total.Bsmt.SF      -8.616873e+00
Condition.1Artery:Central.AirY      2.008637e+03
Condition.1Feedr:Central.AirY       5.238952e+02
Condition.1Park:Central.AirY       -1.816071e+04
Condition.1Rail:Central.AirY        5.077456e+03
Condition.1Artery:log(X12.SF)      -8.479033e+03
Condition.1Feedr:log(X12.SF)       -5.340168e+03
Condition.1Park:log(X12.SF)         6.311889e+03
Condition.1Rail:log(X12.SF)         7.236432e+03
Condition.1Artery:Baths            -2.724680e+02
Condition.1Feedr:Baths             -8.790418e+01
Condition.1Park:Baths              -5.141049e+03
Condition.1Rail:Baths              -6.496596e+03
Condition.1Artery:Kitchen.AbvGr     3.360335e+02
Condition.1Feedr:Kitchen.AbvGr     -4.677704e+03
Condition.1Park:Kitchen.AbvGr      -3.921464e+04
Condition.1Rail:Kitchen.AbvGr       6.348843e+04
Condition.1Artery:Kitchen.Qual     -1.062697e+03
Condition.1Feedr:Kitchen.Qual      -4.183542e+03
Condition.1Park:Kitchen.Qual        4.557790e+03
Condition.1Rail:Kitchen.Qual       -9.075642e+03
Condition.1Artery:FunctionalMaj2    .           
Condition.1Feedr:FunctionalMaj2     .           
Condition.1Park:FunctionalMaj2      .           
Condition.1Rail:FunctionalMaj2      .           
Condition.1Artery:FunctionalMin1    .           
Condition.1Feedr:FunctionalMin1    -1.316042e+03
Condition.1Park:FunctionalMin1     -1.216045e+05
Condition.1Rail:FunctionalMin1      6.237964e+04
Condition.1Artery:FunctionalMin2    .           
Condition.1Feedr:FunctionalMin2     1.019535e+05
Condition.1Park:FunctionalMin2     -9.902724e+03
Condition.1Rail:FunctionalMin2      5.112224e+04
Condition.1Artery:FunctionalMod    -2.491374e+04
Condition.1Feedr:FunctionalMod      4.215363e+04
Condition.1Park:FunctionalMod      -6.379925e+04
Condition.1Rail:FunctionalMod       .           
Condition.1Artery:FunctionalTyp    -4.435788e+03
Condition.1Feedr:FunctionalTyp      3.851065e+04
Condition.1Park:FunctionalTyp      -2.570331e+03
Condition.1Rail:FunctionalTyp       6.733929e+04
Condition.1Artery:Fireplaces       -4.121435e+03
Condition.1Feedr:Fireplaces        -4.367961e+03
Condition.1Park:Fireplaces         -9.981571e+03
Condition.1Rail:Fireplaces         -5.440808e+03
Condition.1Artery:Garage.Cars       8.416206e+02
Condition.1Feedr:Garage.Cars       -7.429902e+01
Condition.1Park:Garage.Cars        -5.794487e+03
Condition.1Rail:Garage.Cars        -2.175790e+04
Condition.1Artery:Paved.DriveP     -4.904995e+04
Condition.1Feedr:Paved.DriveP      -1.291059e+04
Condition.1Park:Paved.DriveP        .           
Condition.1Rail:Paved.DriveP        .           
Condition.1Artery:Paved.DriveY     -1.937015e+04
Condition.1Feedr:Paved.DriveY      -5.171342e+03
Condition.1Park:Paved.DriveY        .           
Condition.1Rail:Paved.DriveY       -1.218357e+04
Overall.Qual:Overall.Cond           5.824994e+02
Overall.Qual:HouseAge               4.192631e+00
Overall.Qual:FoundationCBlock       1.800083e+03
Overall.Qual:FoundationPConc       -1.150747e+03
Overall.Qual:FoundationSlab         7.238879e+03
Overall.Qual:FoundationStone        9.019059e+02
Overall.Qual:FoundationWood         .           
Overall.Qual:Total.Bsmt.SF          1.028472e+01
Overall.Qual:Central.AirY          -3.763103e+02
Overall.Qual:log(X12.SF)            1.457402e+03
Overall.Qual:Baths                  3.018332e+03
Overall.Qual:Kitchen.AbvGr         -1.145279e+04
Overall.Qual:Kitchen.Qual           9.746687e+02
Overall.Qual:FunctionalMaj2         1.292167e+03
Overall.Qual:FunctionalMin1        -1.025699e+04
Overall.Qual:FunctionalMin2        -1.039224e+04
Overall.Qual:FunctionalMod          .           
Overall.Qual:FunctionalTyp         -1.136515e+04
Overall.Qual:Fireplaces             4.013016e+02
Overall.Qual:Garage.Cars            2.219854e+03
Overall.Qual:Paved.DriveP          -3.952631e+03
Overall.Qual:Paved.DriveY          -6.072074e+03
Overall.Cond:HouseAge               5.169968e+01
Overall.Cond:FoundationCBlock      -2.811838e+03
Overall.Cond:FoundationPConc       -1.841508e+03
Overall.Cond:FoundationSlab         4.795250e+03
Overall.Cond:FoundationStone        .           
Overall.Cond:FoundationWood        -3.942050e+03
Overall.Cond:Total.Bsmt.SF          3.769156e+00
Overall.Cond:Central.AirY          -3.003502e+03
Overall.Cond:log(X12.SF)           -1.032565e+03
Overall.Cond:Baths                 -5.497586e+02
Overall.Cond:Kitchen.AbvGr          8.738990e+03
Overall.Cond:Kitchen.Qual           7.617683e+02
Overall.Cond:FunctionalMaj2         .           
Overall.Cond:FunctionalMin1         1.128087e+03
Overall.Cond:FunctionalMin2         1.111867e+04
Overall.Cond:FunctionalMod          1.203638e+04
Overall.Cond:FunctionalTyp          1.052922e+04
Overall.Cond:Fireplaces             1.356432e+03
Overall.Cond:Garage.Cars           -1.863690e+02
Overall.Cond:Paved.DriveP           5.656957e+03
Overall.Cond:Paved.DriveY           3.370411e+03
HouseAge:FoundationCBlock          -1.182097e+02
HouseAge:FoundationPConc           -4.074844e+02
HouseAge:FoundationSlab             3.469731e+02
HouseAge:FoundationStone            .           
HouseAge:FoundationWood             2.817590e+03
HouseAge:Total.Bsmt.SF             -9.247162e-02
HouseAge:Central.AirY               2.637132e+02
HouseAge:log(X12.SF)               -6.219200e+01
HouseAge:Baths                     -7.023008e+01
HouseAge:Kitchen.AbvGr             -2.126380e+02
HouseAge:Kitchen.Qual               6.185930e+01
HouseAge:FunctionalMaj2             .           
HouseAge:FunctionalMin1             4.661907e+01
HouseAge:FunctionalMin2             2.051704e+02
HouseAge:FunctionalMod             -4.016333e+02
HouseAge:FunctionalTyp             -6.113925e+00
HouseAge:Fireplaces                 1.846116e+01
HouseAge:Garage.Cars                5.747629e+01
HouseAge:Paved.DriveP               2.460558e+02
HouseAge:Paved.DriveY               8.509803e+01
FoundationCBlock:Total.Bsmt.SF     -1.434188e+01
FoundationPConc:Total.Bsmt.SF      -1.493536e+01
FoundationSlab:Total.Bsmt.SF        .           
FoundationStone:Total.Bsmt.SF       1.337291e-03
FoundationWood:Total.Bsmt.SF       -1.803278e+01
FoundationCBlock:Central.AirY       1.813977e+03
FoundationPConc:Central.AirY        1.120615e+04
FoundationSlab:Central.AirY        -1.117415e+03
FoundationStone:Central.AirY        1.438219e+04
FoundationWood:Central.AirY        -1.136530e+03
FoundationCBlock:log(X12.SF)       -2.745992e+03
FoundationPConc:log(X12.SF)         3.079723e+03
FoundationSlab:log(X12.SF)          .           
FoundationStone:log(X12.SF)         .           
FoundationWood:log(X12.SF)          .           
FoundationCBlock:Baths             -1.380775e+03
FoundationPConc:Baths              -4.688492e+03
FoundationSlab:Baths                1.877183e+04
FoundationStone:Baths               .           
FoundationWood:Baths               -2.908866e+03
FoundationCBlock:Kitchen.AbvGr      4.762548e+03
FoundationPConc:Kitchen.AbvGr       2.511972e+04
FoundationSlab:Kitchen.AbvGr       -4.321421e+04
FoundationStone:Kitchen.AbvGr      -1.272784e+03
FoundationWood:Kitchen.AbvGr        .           
FoundationCBlock:Kitchen.Qual       1.348843e+03
FoundationPConc:Kitchen.Qual       -2.801662e+03
FoundationSlab:Kitchen.Qual        -4.243940e+03
FoundationStone:Kitchen.Qual        .           
FoundationWood:Kitchen.Qual        -1.553436e+03
FoundationCBlock:FunctionalMaj2     .           
FoundationPConc:FunctionalMaj2      .           
FoundationSlab:FunctionalMaj2       .           
FoundationStone:FunctionalMaj2      .           
FoundationWood:FunctionalMaj2       .           
FoundationCBlock:FunctionalMin1     7.498815e+04
FoundationPConc:FunctionalMin1      5.454084e+04
FoundationSlab:FunctionalMin1       1.666718e+05
FoundationStone:FunctionalMin1      .           
FoundationWood:FunctionalMin1       .           
FoundationCBlock:FunctionalMin2    -1.302539e+04
FoundationPConc:FunctionalMin2     -3.421082e+04
FoundationSlab:FunctionalMin2       3.799190e+04
FoundationStone:FunctionalMin2      4.844266e+04
FoundationWood:FunctionalMin2       .           
FoundationCBlock:FunctionalMod     -1.512480e+04
FoundationPConc:FunctionalMod      -8.024713e+04
FoundationSlab:FunctionalMod       -1.000125e+04
FoundationStone:FunctionalMod       .           
FoundationWood:FunctionalMod        .           
FoundationCBlock:FunctionalTyp      3.516194e+01
FoundationPConc:FunctionalTyp      -2.679816e+04
FoundationSlab:FunctionalTyp        5.383759e+04
FoundationStone:FunctionalTyp      -2.743973e+04
FoundationWood:FunctionalTyp        .           
FoundationCBlock:Fireplaces         4.199150e+02
FoundationPConc:Fireplaces          2.394544e+03
FoundationSlab:Fireplaces           1.697975e+04
FoundationStone:Fireplaces          .           
FoundationWood:Fireplaces           .           
FoundationCBlock:Garage.Cars        1.683153e+03
FoundationPConc:Garage.Cars         6.799961e+02
FoundationSlab:Garage.Cars         -8.313840e+03
FoundationStone:Garage.Cars         .           
FoundationWood:Garage.Cars          .           
FoundationCBlock:Paved.DriveP      -4.517045e+03
FoundationPConc:Paved.DriveP       -3.192248e+03
FoundationSlab:Paved.DriveP         .           
FoundationStone:Paved.DriveP        .           
FoundationWood:Paved.DriveP         .           
FoundationCBlock:Paved.DriveY       6.053565e+03
FoundationPConc:Paved.DriveY        8.751293e+03
FoundationSlab:Paved.DriveY        -3.104708e+04
FoundationStone:Paved.DriveY        .           
FoundationWood:Paved.DriveY         .           
Total.Bsmt.SF:Central.AirY         -1.757439e+01
Total.Bsmt.SF:log(X12.SF)          -4.125011e+00
Total.Bsmt.SF:Baths                 1.167554e+01
Total.Bsmt.SF:Kitchen.AbvGr        -2.549120e+01
Total.Bsmt.SF:Kitchen.Qual          7.871198e-01
Total.Bsmt.SF:FunctionalMaj2       -2.859242e+01
Total.Bsmt.SF:FunctionalMin1       -1.685976e+01
Total.Bsmt.SF:FunctionalMin2       -2.520450e+01
Total.Bsmt.SF:FunctionalMod        -6.850826e+01
Total.Bsmt.SF:FunctionalTyp        -3.544242e+01
Total.Bsmt.SF:Fireplaces            2.491681e+00
Total.Bsmt.SF:Garage.Cars          -5.130345e+00
Total.Bsmt.SF:Paved.DriveP         -1.595442e+01
Total.Bsmt.SF:Paved.DriveY          1.518944e+00
Central.AirY:log(X12.SF)            1.369315e+03
Central.AirY:Baths                 -2.410431e+03
Central.AirY:Kitchen.AbvGr          1.451956e+04
Central.AirY:Kitchen.Qual           6.651701e+03
Central.AirY:FunctionalMaj2         9.314278e+03
Central.AirY:FunctionalMin1         .           
Central.AirY:FunctionalMin2         2.197726e+04
Central.AirY:FunctionalMod          5.992600e+04
Central.AirY:FunctionalTyp          5.616262e+04
Central.AirY:Fireplaces            -5.524425e+03
Central.AirY:Garage.Cars           -5.513441e+02
Central.AirY:Paved.DriveP           1.888029e+04
Central.AirY:Paved.DriveY           1.180651e+04
log(X12.SF):Baths                   3.742547e+03
log(X12.SF):Kitchen.AbvGr          -3.526215e+03
log(X12.SF):Kitchen.Qual           -9.314317e+02
log(X12.SF):FunctionalMaj2          9.523529e+03
log(X12.SF):FunctionalMin1         -2.911798e+03
log(X12.SF):FunctionalMin2          9.829218e+02
log(X12.SF):FunctionalMod           1.998501e+03
log(X12.SF):FunctionalTyp           1.161523e+02
log(X12.SF):Fireplaces              1.554477e+03
log(X12.SF):Garage.Cars            -1.430209e+03
log(X12.SF):Paved.DriveP            1.580954e+02
log(X12.SF):Paved.DriveY            2.550142e+03
Baths:Kitchen.AbvGr                -5.982323e+03
Baths:Kitchen.Qual                  3.782446e+02
Baths:FunctionalMaj2                .           
Baths:FunctionalMin1               -1.761324e+04
Baths:FunctionalMin2               -2.030803e+04
Baths:FunctionalMod                -2.186861e+04
Baths:FunctionalTyp                -2.675980e+04
Baths:Fireplaces                   -4.663751e+03
Baths:Garage.Cars                   2.040156e+03
Baths:Paved.DriveP                  4.924447e+02
Baths:Paved.DriveY                 -9.064070e+02
Kitchen.AbvGr:Kitchen.Qual         -1.007154e+04
Kitchen.AbvGr:FunctionalMaj2        .           
Kitchen.AbvGr:FunctionalMin1        1.683451e+04
Kitchen.AbvGr:FunctionalMin2       -9.872511e+03
Kitchen.AbvGr:FunctionalMod         6.814211e+04
Kitchen.AbvGr:FunctionalTyp         6.592839e+04
Kitchen.AbvGr:Fireplaces           -1.686283e+00
Kitchen.AbvGr:Garage.Cars           2.600280e+03
Kitchen.AbvGr:Paved.DriveP          1.143468e+04
Kitchen.AbvGr:Paved.DriveY          1.344158e+04
Kitchen.Qual:FunctionalMaj2         9.348984e+02
Kitchen.Qual:FunctionalMin1         .           
Kitchen.Qual:FunctionalMin2        -7.266111e+03
Kitchen.Qual:FunctionalMod          .           
Kitchen.Qual:FunctionalTyp          9.218779e+02
Kitchen.Qual:Fireplaces             1.266550e+03
Kitchen.Qual:Garage.Cars            1.718522e+03
Kitchen.Qual:Paved.DriveP           .           
Kitchen.Qual:Paved.DriveY          -5.191888e+03
FunctionalMaj2:Fireplaces          -5.996386e+04
FunctionalMin1:Fireplaces          -1.600998e+04
FunctionalMin2:Fireplaces          -1.775667e+04
FunctionalMod:Fireplaces           -1.957023e+00
FunctionalTyp:Fireplaces           -1.925056e+04
FunctionalMaj2:Garage.Cars          .           
FunctionalMin1:Garage.Cars         -2.172384e+03
FunctionalMin2:Garage.Cars          2.463296e+03
FunctionalMod:Garage.Cars           .           
FunctionalTyp:Garage.Cars          -6.756674e+03
FunctionalMaj2:Paved.DriveP         .           
FunctionalMin1:Paved.DriveP         .           
FunctionalMin2:Paved.DriveP         4.215951e+03
FunctionalMod:Paved.DriveP         -3.261451e+04
FunctionalTyp:Paved.DriveP          4.508108e+03
FunctionalMaj2:Paved.DriveY         .           
FunctionalMin1:Paved.DriveY        -4.465952e+04
FunctionalMin2:Paved.DriveY        -1.222873e+04
FunctionalMod:Paved.DriveY         -2.954323e+04
FunctionalTyp:Paved.DriveY          2.005576e+00
Fireplaces:Garage.Cars              1.111989e+03
Fireplaces:Paved.DriveP             1.242086e+04
Fireplaces:Paved.DriveY             1.161963e+04
Garage.Cars:Paved.DriveP            6.117077e+02
Garage.Cars:Paved.DriveY           -2.528726e+03
Bsmt.YN:Bsmt.ExposureNone           .           
Bsmt.YN:Bsmt.Exposure              -4.268130e+03
Bsmt.YN:Bsmt.ExposureAv             2.281787e+04
Bsmt.YN:Bsmt.ExposureGd             3.010838e+04
Bsmt.YN:Bsmt.ExposureMn             1.791112e+04
Bsmt.YN:Bsmt.ExposureNo             1.849160e+04
housing.blasso.RJ = blasso(X.scaled, Y, RJ = TRUE, verb=0)
Error in blasso(X.scaled, Y, RJ = TRUE, verb = 0) : 
  NA/NaN/Inf in foreign function call (arg 5)
# center and scale X
X.scaled = X
X.scaled[,2:15] = scale(X[,2:15], center =college.ridge.cv$xm[c(-1,-16)],
                        scale = college.ridge.cv$scales[c(-1,-16)])
                        
# RJ = FALSE
# college.blasso = blasso(X.scaled, Y, RJ = FALSE, verb=0) 
# ypred.blasso = mean(college.blasso$mu) +
#                as.matrix(test.scaled) %*% apply(college.blasso$beta, 2, mean) 
# rmse.blasso = rmse(exp(ctest$Apps), exp(ypred.blasso))
# model selection with RJ = TRUE
college.blasso.RJ = blasso(X.scaled, Y, RJ = TRUE, verb=0)
ypred.blasso.RJ = mean(college.blasso.RJ$mu) + as.matrix(test.scaled) %*% apply(college.blasso.RJ$beta, 2, mean)
rmse.blasso.RJ = rmse(exp(ctest$Apps), exp(ypred.blasso.RJ)) 
summary(college.blasso.RJ)$coef
model.lasso = glmnet(X.train, log(data_train$price), alpha=1)
cv.lasso = cv.glmnet(X.train, log(data_train$price), alpha=1)
yhat.lasso = exp(predict(model.lasso, s=cv.lasso$lambda.min, type="response", newx = X.test))

sqrt(mean(((yhat.lasso)-data_test$price)^2))


# Ridge for the simplified model (from BIC)

model.ridge = glmnet(X.train, log(data_train$price), alpha=0)
cv.ridge = cv.glmnet(X.train, log(data_train$price), alpha=0)
yhat.ridge = predict(model.ridge, s=cv.ridge$lambda.min, type="response", newx = X.test)

sqrt(mean((exp(yhat.ridge)-data_test$price)^2))

Part III: Write Up

Once you are satisfied with your model, provide a write up of your data analysis project in a new Rmd file/pdf file: writeup.Rmd by copying over salient parts of your R notebook. The written assignment consists of five parts:

  1. Exploratory data analysis (20 points): must include three correctly labeled graphs and an explanation that highlight the most important features that went into your model building.

  2. Development and assessment of an initial model from Part I (10 points)

  • Initial model: must include a summary table and an explanation/discussion for variable selection. Interpretation of coefficients desirable for full points.

  • Model selection: must include a discussion

  • Residual: must include a residual plot and a discussion

  • RMSE: must include an RMSE and an explanation (other criteria desirable)

  • Model testing: must include an explanation

  1. Development of the final model (20 points)
  • Final model: must include a summary table

  • Variables: must include an explanation

  • Variable selection/shrinkage: must use appropriate method and include an explanation

  1. Assessment of the final model (25 points)
  • Residual: must include a residual plot and a discussion

  • RMSE: must include an RMSE and an explanation (other criteria desirable)

  • Model evaluation: must include an evaluation discussion

  • Model testing : must include a discussion

  • Model result: must include a selection of the top 10 undervalued and overvalued houses

  1. Conclusion (10 points): must include a summary of results and a discussion of things learned

Part IV

Create predictions for the validation data from your final model and write out to a file prediction-validation.Rdata This should have the same format as the models in Part I and II.

10 points

Class Presentations

Each Group should prepare 5 slides in their Github repo: (save as slides.pdf)

  • Most interesting graphic (a picture is worth a thousand words prize!)

  • Best Model (motivation, how you found it, why you think it is best)

  • Best Insights into predicting Sales Price.

  • 2 Best Houses to purchase (and why)

  • Best Team Name/Graphic

We will select winners based on the above criteria and overall performance.

Finally your repo should have: writeup.Rmd, writeup.pdf, slides.Rmd (and whatever output you use for the presentation) and predict.Rdata and predict-validation.Rdata.

LS0tCnRpdGxlOiAiRmluYWwgRGF0YSBBbmFseXNpcyBQcm9qZWN0IgpkYXRlOiAgIldyaXRlIHVwIGR1ZSBBcHJpbCAyOCBhdCA1IHBtIgpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sKLS0tCgpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGVjaG8gPSBUUlVFKQpgYGAKCgpGb3IgdGhpcyBwcm9qZWN0IHlvdSB3aWxsIHRha2UgdGhlIHJvbGUgb2YgYSBjb25zdWx0YW50IGhpcmVkIGJ5IGEgcmVhbCBlc3RhdGUgaW52ZXN0bWVudCBmaXJtIGluIEFtZXMsIElvd2EsIGEgbWlkLXdlc3QgdG93biBpbiB0aGUgVW5pdGVkIFN0YXRlcywgdG8gYW5hbHl6ZSBkYXRhIGluIG9yZGVyIHRvIGhlbHAgcHJvdmlkZSBpbnNpZ2h0IGludG8gaG93IHRoZSBmaXJtIHNob3VsZCBpbnZlc3QgZm9yIGhpZ2hlc3QgcHJvZml0cywgYW5kIHRvIHF1YW50aWZ5IGFuZCBjb21tdW5pY2F0ZSB0byB0aGUgY29tcGFueSBtYW5hZ2VtZW50IHdoYXQgdHlwZXMgb2YgcmVhbCBlc3RhdGUgcHJvcGVydGllcyBhcmUgZ29vZCBpbnZlc3RtZW50cyBhbmQgd2h5LiBUaGV5IGhhdmUgcHJvdmlkZWQgeW91IHdpdGggZGF0YSBvbiBob3VzaW5nIHNhbGVzIGZyb20gYmV0d2VlbiAyMDA2IHRvIDIwMTAgdGhhdCBjb250YWlucyBpbmZvcm1hdGlvbiBhYm91dCB0aGUgY2hhcmFjdGVyaXN0aWNzIG9mIHRoZSBob3VzZSAobnVtYmVyIG9mIGJlZHJvb21zLCBudW1iZXIgb2YgYmF0aHJvb21zLCBzcXVhcmUgZm9vdGFnZSwgZXRjLikgYW5kIHRoZSBob3VzZSdzIHNhbGUgcHJpY2UuIFRoZSBjb2RlYm9vayBmb3IgdGhpcyBkYXRhIHNldCBpcyBbYXZhaWxhYmxlIG9ubGluZSBoZXJlIF0oaHR0cHM6Ly93dzIuYW1zdGF0Lm9yZy9wdWJsaWNhdGlvbnMvanNlL3YxOW4zL2RlY29jay9kYXRhZG9jdW1lbnRhdGlvbi50eHQpICBvciBpbiB0aGUgRGF0YSBmb2xkZXIgaW4geW91ciByZXBvLgoKIyMgQWJvdXQgdGhlIERhdGEgQW5hbHlzaXMgUHJvamVjdAoKSXQncyBnZW5lcmFsbHkgYSBiYWQgaWRlYSB0byBidXkgdGhlIG1vc3QgZXhwZW5zaXZlIGhvdXNlIGluIHRoZSBuZWlnaGJvcmhvb2QuIEFuZCByZW1lbWJlciB0aGUgcmVhbCBlc3RhdGUgYWdlbnRzJyBtYW50cmE6IExvY2F0aW9uLCBsb2NhdGlvbiwgbG9jYXRpb24hIEtlZXAgaW4gbWluZCB0aGF0IHRoZSBnb2FsIGlzIHRvIG1ha2UgbW9uZXkgZm9yIHlvdXIgaW52ZXN0b3JzLCBhbmQgaGVuY2UgaW52ZXN0aW5nIGluIGEgcHJvcGVydHkgdGhhdCBpcyBvdmVydmFsdWVkIChjb3N0aW5nIG1vcmUgdGhhbiBpdCBpcyB3b3J0aCkgaXMgcmFyZWx5IGEgZ29vZCBpZGVhLiBUaGlzIG1lYW5zIHRoYXQgaXQncyBjcml0aWNhbCB0byBrbm93IHdoaWNoIHByb3BlcnRpZXMgYXJlIG92ZXJ2YWx1ZWQgYW5kIHdoaWNoIGFyZSB1bmRlcnZhbHVlZC4gIFRoZSBjb21wYW55IHRoYXQgaGlyZWQgeW91IGhhcyBtYW55IHF1ZXN0aW9ucyBmb3IgeW91IGFib3V0IHRoZSBob3VzaW5nIG1hcmtldC4gSXQgaXMgdXAgdG8geW91IHRvIGRlY2lkZSB3aGF0IG1ldGhvZHMgeW91IHdhbnQgdG8gdXNlIChmcmVxdWVudGlzdCBvciBCYXllc2lhbikgdG8gYW5zd2VyIHRoZXNlIHF1ZXN0aW9ucywgYW5kIGltcGxlbWVudCB0aGVtIHRvIGhlbHAgdG8gaWRlbnRpZnkgdW5kZXJ2YWx1ZWQgYW5kIG92ZXJ2YWx1ZWQgcHJvcGVydGllcy4KCgpZb3Ugd2lsbCBoYXZlIHRocmVlIGRhdGEgc2V0czogYSBzdWJzZXQgZm9yIHRyYWluaW5nLCBhIHN1YnNldCBmb3IgdGVzdGluZywgYW5kIGEgdGhpcmQgc3Vic2V0IGZvciB2YWxpZGF0aW9uLiBZb3Ugd2lsbCBiZSBhc2tlZCB0byBkbyBkYXRhIGV4cGxvcmF0aW9uIGFuZCBidWlsZCB5b3VyIG1vZGVsIChvciBtb2RlbHMpIGluaXRpYWxseSB1c2luZyBvbmx5IHRoZSB0cmFpbmluZyBkYXRhLiBUaGVuLCB5b3Ugd2lsbCB0ZXN0IHlvdXIgbW9kZWwgb24gdGhlIHRlc3RpbmcgZGF0YSwgYW5kIGZpbmFsbHkgdmFsaWRhdGUgdXNpbmcgdGhlIHZhbGlkYXRpb24gZGF0YS4gV2UgYXJlIGNoYWxsZW5naW5nIHlvdSB0byBrZWVwIHlvdXIgYW5hbHlzaXMgZXhwZXJpZW5jZSByZWFsaXN0aWMsIGFuZCBpbiBhIHJlYWxpc3RpYyBzY2VuYXJpbyB5b3Ugd291bGQgbm90IGhhdmUgYWNjZXNzIHRvIGFsbCB0aHJlZSBvZiB0aGVzZSBkYXRhIHNldHMgYXQgb25jZS4gIFlvdSB3aWxsIGJlIGFibGUgdG8gc2VlIG9uIG91ciBzY29yZWJvYXJkIGhvdyB3ZWxsIHlvdXIgdGVhbSBpcyBkb2luZyBiYXNlZCBvbiBpdHMgcHJlZGljdGl2ZSBwZXJmb3JtYW5jZSBvbiB0aGUgdGVzdGluZyBkYXRhLiAgQWZ0ZXIgeW91ciBwcm9qZWN0IGlzIHR1cm5lZCBpbiB5b3Ugd2lsbCBzZWUgdGhlIGZpbmFsIHNjb3JlIG9uIHRoZSB2YWxpZGF0aW9uIHNldC4KCkFsbCBtZW1iZXJzIG9mIHRoZSB0ZWFtIHNob3VsZCBjb250cmlidXRlIGVxdWFsbHkgYW5kIGFuc3dlciBhbnkgcXVlc3Rpb25zIGFib3V0IHRoZSBhbmFseXNpcyBhdCB0aGUgZmluYWwgcHJlc2VudGF0aW9uLgoKRm9yIHlvdXIgYW5hbHlzaXMgY3JlYXRlIGEgbmV3IG5vdGVib29rIG5hbWVkICJwcm9qZWN0LlJtZCIKYW5kIHVwZGF0ZSBhY2NvcmRpbmdseSByYXRoZXIgdGhhbiBlZGl0aW5nIHRoaXMuCgoKIyMjIFJlYWQgaW4gVHJhaW5pbmcgRGF0YQoKVG8gZ2V0IHN0YXJ0ZWQgcmVhZCBpbiB0aGUgdHJhaW5pbmcgZGF0YS4gCmBgYHtyIHJlYWQtZGF0YX0KbGlicmFyeShkcGx5cikKbGlicmFyeSh0aWR5cikKbG9hZCgiYW1lc190cmFpbi5SZGF0YSIpCgpwcmludChwYXN0ZTAoIlRoZSBkYXRhc2V0IGhhcyAiLCBkaW0oYW1lc190cmFpbilbMV0sICIgbnVtYmVyIG9mIG9ic2VydmF0aW9ucyBhbmQgIiwgZGltKGFtZXNfdHJhaW4pWzJdLCAiIGZlYXR1cmVzIikpCgojVmFyaWFibGVzIHdpdGggTkEncyBhbmQgdGhlaXIgcHJvcG9ydGlvbiBvZiBtaXNzaW5nIGRhdGEKbWlzcyA9IGFwcGx5KGlzLm5hKGFtZXNfdHJhaW4pLCAyLCBzdW0pCm1pc3NfcHJvcCA9IHJvdW5kKG1pc3NbbWlzcz4wXS9ucm93KGFtZXNfdHJhaW4pLCAzKQpwcmludChtaXNzX3Byb3ApIAp3aGljaChtaXNzX3Byb3A+MC41KSAjIGZvdXIgZmVhdHVyZXMgaGF2ZSBncmVhdGVyIHRoYW4gNTAlIG9mIGRhdGEgIm1pc3NpbmciIC0tIGRyb3AgdGhlc2UgdmFyaWFibGVzCmBgYAoKCk5vdGVzIGFib3V0IGRhdGEgY2xlYW5pbmc6CgpXZSBkcm9wcGVkIOKAnHV0aWxpdGllc+KAnSAodHlwZSBvZiB1dGlsaXRpZXMgYXZhaWxhYmxlKSBzaW5jZSBpbiB0aGUgdHJhaW5pbmcgc2V0LCBvbmx5IDIgb2JzZXJ2YXRpb25zIGRpZCBub3QgaGF2ZSBhbGwgdGhlIHV0aWxpdGllcyAoZWxlY3RyaWNpdHksIGdhcywgd2F0ZXIgYW5kIHNld2FnZSkuIEludHVpdGl2ZWx5LCBtb3N0IG1vZGVybiBwcm9wZXJ0eSBhcmUgZXF1aXBwZWQgd2l0aCB0aGVzZSBiYXNpYyBwdWJsaWMgdXRpbGl0aWVzIGFuZCBrZWVwaW5nIHRoZSB2YXJpYWJsZSB3b3VsZCB0aGVyZWZvcmUgYmUgdW5uZWNlc3NhcnkuIAoKV2UgYWxzbyBkcm9wcGVkIOKAnGNvbmRpdGlvbiAy4oCdIChwcm94aW1pdHkgdG8gdmFyaW91cyBjb25kaXRpb25zIGlmIG1vcmUgdGhhbiBvbmUgaXMgcHJlc2VudCkgc2luY2UgaXQgc2VlbWVkIHJlZHVuZGFudCBmcm9tIG91ciB0cmFpbmluZyBzZXQuIEdpdmVuIENvbmRpdGlvbiAxLCBvbmx5IDEyIHByb3BlcnRpZXMgd2VyZSBub3QgY2xvc2UgdG8gbm9ybWFsIGNvbmRpdGlvbnMuIAoKSW4gdGhlIG9yaWdpbmFsIHNjYWxlLCAxOTkwIGFuZCAxOTAwIHdvdWxkIG5vdCBiZSBtdWNoIGRpZmZlcmVudC4gVGhlcmVmb3JlLCB3ZSBjaGFuZ2VkIHRoZSBzY2FsZSB0byB0aGUgbnVtYmVyIG9mIHllYXJzIHNpbmNlIGxhc3QgY29uc3RydWN0aW9uIG9yIHJlbW9kZWxsaW5nLCBzdWJ0cmFjdGVkIGZyb20geWVhciAyMDEwICh0aGUgZW5kIHllYXIgaW4gdGhlIGRhdGFzZXQpLiAKCkFub3RoZXIgdmFyaWFibGUgZHJvcHBlZCBpbiBvdXIgbW9kZWwgd2FzIOKAnHJvb2YgbWF0ZXJpYWzigJ0gc2luY2Ugb25seSAxJSBvZiB0aGUgcHJvcGVydHkgdXNlZCBtYXRlcmlhbCBvdGhlciB0aGFuIOKAnHN0YW5kYXJkIGNvbXBvc2l0ZSBzaGluZ2xl4oCdLiBTaW1pbGFybHksIOKAnGhlYXRpbmfigJ0gd2FzIGFsc28gZHJvcHBlZCBzaW5jZSBtb3JlIHRoYW4gOTUlIG9mIHRoZSBwcm9wZXJ0eSBoYXMgZ2FzIGZvcmNlZCB3YXJtIGFpciBmdXJuYWNlIChHYXNBKSBpbnN0ZWFkIG9mIG90aGVyIHR5cGVzIG9mIGhlYXRpbmcuCgpGb3Ig4oCcZXh0ZXJpb3IgcXVhbGl0eeKAnSBhbmQg4oCcZXh0ZXJpb3IgY29uZGl0aW9u4oCdCSwgd2UgcmVjb2RlZCB0aGVzZSBvcmRpbmFsIHZhcmlhYmxlcyB0byAxLTUgdG8gcmVwbGFjZSB0aGUgb3JpZ2luYWwgc2NhbGUgb2YgY29uZGl0aW9ucyAoZnJvbSBwb29yIHRvIGV4Y2VsbGVudCkuIFNpbWlsYXJseSwgd2UgcmVjb2RlZCDigJxiYXNlbWVudCBleHBvc3VyZeKAnSBhbmQg4oCcYmFzZW1lbnQgcmF0aW5n4oCdIGV4Y2VwdCB0aGF0IHRoZSBuZXcgc2NhbGUgd291bGQgc3RhcnQgZnJvbSAwIGZvciBwcm9wZXJ0aWVzIHdpdGhvdXQgYmFzZW1lbnQuIAoJCQkJCQkJCQkJCQkKRXh0ZXIgUXVhbCAoT3JkaW5hbCk6IEV2YWx1YXRlcyB0aGUgcXVhbGl0eSBvZiB0aGUgbWF0ZXJpYWwgb24gdGhlIGV4dGVyaW9yCgkJCQkJCQogICAgICBFeCAgICAgICBFeGNlbGxlbnQKICAgICAgIEdkICAgICAgIEdvb2QKICAgICAgIFRBICAgICAgIEF2ZXJhZ2UvVHlwaWNhbAogICAgICAgRmEgICAgICAgRmFpcgogICAgICAgUG8gICAgICAgUG9vcgoKCgkJCQkJCQpFeHRlciBDb25kIChPcmRpbmFsKTogRXZhbHVhdGVzIHRoZSBwcmVzZW50IGNvbmRpdGlvbiBvZiB0aGUgbWF0ZXJpYWwgb24gdGhlIGV4dGVyaW9yCgogICAgICBFeCAgICAgICBFeGNlbGxlbnQKICAgICAgIEdkICAgICAgIEdvb2QKICAgICAgIFRBICAgICAgIEF2ZXJhZ2UvVHlwaWNhbAogICAgICAgRmEgICAgICAgRmFpcgogICAgICAgUG8gICAgICAgUG9vcgoKCkNvbnRpbnVvdXMgdmFyaWFibGVzIHN1Y2ggYXMg4oCcMXN0IGZsb29yIHNxdWFyZSBmZWV04oCdIGFuZCDigJwybmQgZmxvb3Igc3F1YXJlIGZlZXTigJ0gd2VyZSBsb2ctdHJhbnNmb3JtZWQgZm9yIGludGVycHJldGF0aW9uIHB1cnBvc2UuIAoKRm9yIHZhcmlhYmxlIOKAnGZ1bmN0aW9uYWzigJ0sIHdlIHJlY29kZWQgZGlmZmVyZW50IG9yZGluYWwgbGV2ZWxzIGludG8gYmluYXJ5IGxldmVscyAtLS0gdHlwaWNhbCBmdW5jdGlvbmFsaXR5IG9yIG5vdCwgaW5jbHVkaW5nIG1pbm9yIGFuZCBtYWpvciBkZWR1Y3Rpb25zLiAgCQkKCldlIHN1bW1lZCB1cCB0aGUgbnVtYmVyIG9mIGJhdGhyb29tcyB0byBvbmUgY29udGludW91cyB2YXJpYWJsZS4gTm90ZSB0aGF0IG9uZSBoYWxmLWJhdGhyb29tIHdvdWxkIGJlIGNvZGVkIGFzIDAuNS4JCmBgYHtyfQojIERpZCBub3QgcmVtb3ZlIGFueSBOQSBlbnRyaWVzIGluIExvdC5mcm9udGFnZQoKZGF0YT1hbWVzX3RyYWluCgpkYXRhIDwtIGRhdGEgICU+JQogICNmaWx0ZXIoIWlzLm5hKExvdC5Gcm9udGFnZSkpICU+JSAKICBtdXRhdGUoTVMuU3ViQ2xhc3M9IGZhY3RvcihNUy5TdWJDbGFzcykpICU+JQogIG11dGF0ZShBbGxleSA9ICBmYWN0b3IoQWxsZXksIGxldmVscyA9IGxldmVscyhhZGROQShBbGxleSkpLCBsYWJlbHMgPSBjKGxldmVscyhBbGxleSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEhvdXNlQWdlID0gWXIuU29sZC0gcG1heChZZWFyLkJ1aWx0LCBZZWFyLlJlbW9kLkFkZCkpICAlPiUKICBmaWx0ZXIoIWlzLm5hKE1hcy5WbnIuQXJlYSkpICAlPiUgCiAgbXV0YXRlKEJzbXQuWU4gPSAxKighaXMubmEoQnNtdC5RdWFsKSkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSAgZmFjdG9yKEJzbXQuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuUXVhbCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LlF1YWwpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSByZWxldmVsKEJzbXQuUXVhbCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210LkNvbmQgPSAgZmFjdG9yKEJzbXQuQ29uZCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuQ29uZCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LkNvbmQpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210LkNvbmQgPSByZWxldmVsKEJzbXQuQ29uZCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210LkV4cG9zdXJlID0gIGZhY3RvcihCc210LkV4cG9zdXJlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5FeHBvc3VyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LkV4cG9zdXJlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdC5FeHBvc3VyZSA9IHJlbGV2ZWwoQnNtdC5FeHBvc3VyZSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMT0gIGZhY3RvcihCc210RmluLlR5cGUuMSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4xKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4xKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoQnNtdEZpbi5UeXBlLjEgPSByZWxldmVsKEJzbXRGaW4uVHlwZS4xLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXRGaW4uVHlwZS4yPSAgZmFjdG9yKEJzbXRGaW4uVHlwZS4yLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdEZpbi5UeXBlLjIpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdEZpbi5UeXBlLjIpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMiA9IHJlbGV2ZWwoQnNtdEZpbi5UeXBlLjIsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoWDEyLlNGPSBYMXN0LkZsci5TRisgWDJuZC5GbHIuU0YpICAlPiUgICAKICBmaWx0ZXIoIWlzLm5hKEJzbXQuRnVsbC5CYXRoKSkgJT4lCiAgZmlsdGVyKCFpcy5uYShCc210LkhhbGYuQmF0aCkpICU+JQogIG11dGF0ZShCYXRocyA9IEJzbXQuRnVsbC5CYXRoICsgMC41KkJzbXQuSGFsZi5CYXRoICsgRnVsbC5CYXRoICsgMC41KkhhbGYuQmF0aCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5ZTiA9IDEqKEZpcmVwbGFjZXM+MCkpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuUXUgPSAgZmFjdG9yKEZpcmVwbGFjZS5RdSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEZpcmVwbGFjZS5RdSkpLCBsYWJlbHMgPSBjKGxldmVscyhGaXJlcGxhY2UuUXUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuUXUgPSByZWxldmVsKEZpcmVwbGFjZS5RdSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuWU4gPSAxKighaXMubmEoR2FyYWdlLkNvbmQpKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5UeXBlID0gIGZhY3RvcihHYXJhZ2UuVHlwZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5UeXBlKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5UeXBlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLlR5cGUgPSByZWxldmVsKEdhcmFnZS5UeXBlLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5GaW5pc2ggPSAgZmFjdG9yKEdhcmFnZS5GaW5pc2gsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuRmluaXNoKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5GaW5pc2gpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShHYXJhZ2UuRmluaXNoID0gcmVsZXZlbChHYXJhZ2UuRmluaXNoLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5RdWFsID0gIGZhY3RvcihHYXJhZ2UuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5RdWFsKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5RdWFsKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLlF1YWwgPSByZWxldmVsKEdhcmFnZS5RdWFsLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5Db25kID0gIGZhY3RvcihHYXJhZ2UuQ29uZCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5Db25kKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkNvbmQgPSByZWxldmVsKEdhcmFnZS5Db25kLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKFBvcmNoLkFyZWEgPSBXb29kLkRlY2suU0YrIE9wZW4uUG9yY2guU0YrRW5jbG9zZWQuUG9yY2grWDNTc24uUG9yY2ggKyBTY3JlZW4uUG9yY2gpICU+JQogIG11dGF0ZShQb29sLllOID0gMSooUG9vbC5BcmVhPjApKSAlPiUKICBtdXRhdGUoUG9vbC5RQyA9ICBmYWN0b3IoUG9vbC5RQywgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKFBvb2wuUUMpKSwgbGFiZWxzID0gYyhsZXZlbHMoUG9vbC5RQyksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKFBvb2wuUUMgPSByZWxldmVsKFBvb2wuUUMsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoRmVuY2UgPSAgZmFjdG9yKEZlbmNlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmVuY2UpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmVuY2UpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNaXNjLkZlYXR1cmUgPSAgZmFjdG9yKE1pc2MuRmVhdHVyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKE1pc2MuRmVhdHVyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhNaXNjLkZlYXR1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNby5Tb2xkID0gYXMuZmFjdG9yKE1vLlNvbGQpKSAlPiUKICBtdXRhdGUoWXIuU29sZCA9IGFzLmZhY3RvcihZci5Tb2xkKSkgJT4lCiAgZHBseXI6OnNlbGVjdCgtR2FyYWdlLllyLkJsdCkgJT4lCiAgbXV0YXRlKENvbmRpdGlvbi4xID0gYXMuY2hhcmFjdGVyKENvbmRpdGlvbi4xKSkgJT4lCiAgbXV0YXRlKEtpdGNoZW4uUXVhbD1wbHlyOjptYXB2YWx1ZXMoS2l0Y2hlbi5RdWFsLCBmcm9tID0gYygiUG8iLCAiRmEiLCAiVEEiLCJHZCIsICJFeCIgKSwgdG8gPSBjKCIxIiwgIjIiLCAiMyIsICI0IiwgIjUiKSkpICU+JQogIG11dGF0ZShLaXRjaGVuLlF1YWwgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihLaXRjaGVuLlF1YWwpKSkgJT4lCiAgbXV0YXRlKEhlYXRpbmcuUUM9cGx5cjo6bWFwdmFsdWVzKEhlYXRpbmcuUUMsIGZyb20gPSBjKCJQbyIsICJGYSIsICJUQSIsIkdkIiwgIkV4IiApLCB0byA9IGMoIjEiLCAiMiIsICIzIiwgIjQiLCAiNSIpKSkgJT4lCiAgbXV0YXRlKEhlYXRpbmcuUUMgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihIZWF0aW5nLlFDKSkpICU+JQogIG11dGF0ZShCc210LlF1YWwgPSBkcm9wbGV2ZWxzKEJzbXQuUXVhbCkpICU+JQogIG11dGF0ZShGdW5jdGlvbmFsID0gZHJvcGxldmVscyhGdW5jdGlvbmFsKSkgJT4lCiAgbXV0YXRlKFJvb2YuTWF0bCA9IGRyb3BsZXZlbHMoUm9vZi5NYXRsKSkKCiMgU2ltcGxpZnkgQ29uZGl0aW9uIDEgKFBhcmssIFJhaWwsIE5vcm1hbCkKaW5kX3JhaWw8LXdoaWNoKGRhdGEkQ29uZGl0aW9uLjE9PSJSUk5uIiB8IGRhdGEkQ29uZGl0aW9uLjE9PSJSUkFuIiB8IGRhdGEkQ29uZGl0aW9uLjE9PSJSUk5lIiB8IGRhdGEkQ29uZGl0aW9uLjE9PSJSUkFlIikKaW5kX3Bhcms8LXdoaWNoKGRhdGEkQ29uZGl0aW9uLjE9PSJQb3NOIiB8IGRhdGEkQ29uZGl0aW9uLjE9PSJQb3NBIikKZGF0YSRDb25kaXRpb24uMVtpbmRfcmFpbF08LSJSYWlsIgpkYXRhJENvbmRpdGlvbi4xW2luZF9wYXJrXTwtIlBhcmsiCmRhdGEgPSBkYXRhICU+JSAKICBtdXRhdGUoQ29uZGl0aW9uLjEgPSBmYWN0b3IoQ29uZGl0aW9uLjEpKSAlPiUKICBtdXRhdGUoQ29uZGl0aW9uLjEgPSByZWxldmVsKENvbmRpdGlvbi4xLCByZWY9Ik5vcm0iKSkgCgojIEVsaW1pbmF0ZSB0aGUgb25lIGVudHJ5IGluICdFeHBvc3VyZScgdGhhdCBoYWQgYmVlbiBsZWZ0IGNvbXBsZXRlbHkgZW1wdHkKZGF0YV90cmFpbjwtZGF0YQpkYXRhX3RyYWluJEJzbXQuRXhwb3N1cmVbd2hpY2goZGF0YV90cmFpbiRCc210LkV4cG9zdXJlPT0iIildPC0iTm9uZSIKZGF0YV90cmFpbiRCc210LkV4cG9zdXJlPC1kcm9wbGV2ZWxzKGRhdGFfdHJhaW4kQnNtdC5FeHBvc3VyZSkKCmRhdGFfdHJhaW4kUG9vbC5BcmVhPC1kYXRhX3RyYWluJFBvb2wuQXJlYSsxCmRhdGFfdHJhaW4kVG90YWwuQnNtdC5TRjwtZGF0YV90cmFpbiRUb3RhbC5Cc210LlNGKzEKYGBgCgpUaGUgYE5laWdoYm9yaG9vZGAgdmFyaWFibGUsIHR5cGljYWxseSBvZiBsaXR0bGUgaW50ZXJlc3Qgb3RoZXIgdGhhbiB0byBtb2RlbCB0aGUgbG9jYXRpb24gZWZmZWN0LCBtYXkgYmUgb2YgbW9yZSByZWxldmFuY2Ugd2hlbiB1c2VkIHdpdGggdGhlIFttYXBdKGh0dHA6Ly93d3cuYW1zdGF0Lm9yZy9wdWJsaWNhdGlvbnMvanNlL3YxOW4zL2RlY29jay9BbWVzUmVzaWRlbnRpYWwucGRmKS4KCldlIGFyZSByZXN0cmljdGluZyBhdHRlbnRpb24gdG8ganVzdCB0aGUgIm5vcm1hbCBzYWxlcyIgY29uZGl0aW9uLgoKCiMjIFBhcnQgSTogU2ltcGxlIE1vZGVsCgpJbiB0aGUgZmlyc3QgbW9kZWwgeW91IGFyZSBhbGxvd2VkIG9ubHkgbGltaXRlZCBtYW5pcHVsYXRpb25zIG9mIHRoZSBvcmlnaW5hbCBkYXRhIHNldCB0byBwcmVkaWN0IHRoZSBzYWxlcyBwcmljZSBgcHJpY2VgLiBZb3UgYXJlIGFsbG93ZWQgdG8gdGFrZSBwb3dlciB0cmFuc2Zvcm1hdGlvbnMgb2YgdGhlIG9yaWdpbmFsIHZhcmlhYmxlcyBbc3F1YXJlIHJvb3RzLCBsb2dzLCBpbnZlcnNlcywgc3F1YXJlcywgZXRjLl0gYnV0IHlvdSBhcmUgTk9UIGFsbG93ZWQgdG8gY3JlYXRlIGludGVyYWN0aW9uIHZhcmlhYmxlcy4gVGhpcyBtZWFucyB0aGF0IGEgdmFyaWFibGUgbWF5IG9ubHkgYmUgdXNlZCBvbmNlIGluIGFuIGVxdWF0aW9uIFtpZiB5b3UgdXNlICQgeF4yJCBkb27igJl0IHVzZSAkeCRdLiBBZGRpdGlvbmFsbHksIHlvdSBtYXkgZWxpbWluYXRlIGFueSBkYXRhIHBvaW50cyB5b3UgZGVlbSB1bmZpdC4gVGhpcyBtb2RlbCBzaG91bGQgaGF2ZSBhIG1pbmltdW0gci1zcXVhcmUgb2YgNzMlIChpbiB0aGUgb3JpZ2luYWwgdW5pdHMpIGFuZCBjb250YWluIGF0IGxlYXN0IDYgdmFyaWFibGVzIGJ1dCBmZXdlciB0aGFuIDIwLiAgIAoKCgoKYGBge3J9CiMjIyBwZXJmcm9tYW5jZSBldmxhdXRpb24gZnVuY3Rpb24KcGVyZm9ybWFuY2U8LSBmdW5jdGlvbihZLCBZaGF0KXsKICAKICBiaWFzPC0gbWVhbihZLVloYXRbLDFdKQogIAogIG1heC5kZXY8LW1heChhYnMoWS1ZaGF0WywxXSkpCiAgCiAgbWVhbi5kZXY8LW1lYW4oYWJzKFktWWhhdFssMV0pKQogIAogIFJNU0U8LXNxcnQobWVhbigoWS1ZaGF0WywxXSleMikpCiAgCiAgY292ZXJhZ2U8LW1lYW4oKFk+WWhhdFssMl0pICYgKFk8WWhhdFssM10pKQogIAogIG91dDwtZGF0YS5mcmFtZShiaWFzPWJpYXMsIG1heC5kZXY9bWF4LmRldiwgbWVhbi5kZXY9bWVhbi5kZXYsIFJNRVM9Uk1TRSwgQ292ZXJhZ2U9Y292ZXJhZ2UpCiAgCiAgcmV0dXJuKG91dCkKfQpgYGAKCgoKCmBgYHtyfQpsaWJyYXJ5KE1BU1MpCgojIEJhc2UgbW9kZWwgd2l0aCB0cmFuc2Zvcm1lZCBwcmVkaWN0b3JzIAogbW9kZWw9bG0ocHJpY2UgfiBNUy5TdWJDbGFzcyArIE1TLlpvbmluZyArIGxvZyhMb3QuRnJvbnRhZ2UpICsgbG9nKExvdC5BcmVhKSArICBTdHJlZXQgKyBBbGxleSArIExvdC5TaGFwZSArIExhbmQuQ29udG91ciArIExvdC5Db25maWcgKyBMYW5kLlNsb3BlICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgICsgQmxkZy5UeXBlICsgSG91c2UuU3R5bGUgKyBPdmVyYWxsLlF1YWwgKyBPdmVyYWxsLkNvbmQgKyBIb3VzZUFnZSArICAgUm9vZi5TdHlsZSArIFJvb2YuTWF0bCArIEV4dGVyaW9yLjFzdCArIE1hcy5WbnIuVHlwZSArIGxvZygxK01hcy5WbnIuQXJlYSkgKyBFeHRlci5Db25kICsgRXh0ZXIuUXVhbCArICAgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBUb3RhbC5Cc210LlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIEVsZWN0cmljYWwgKyBsb2coWDEyLlNGKSArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuUXUgKyBHYXJhZ2UuVHlwZSArIEdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5Db25kICsgR2FyYWdlLlF1YWwgKyBQYXZlZC5Ecml2ZSArIGxvZygxK1Bvb2wuQXJlYSkgKyBQb29sLlFDICsgRmVuY2UgKyBNaXNjLlZhbCArIE1vLlNvbGQgICtZci5Tb2xkICsgU2FsZS5UeXBlICArIFRvdGFsU3EsIGRhdGE9ZGF0YV90cmFpbikKCgojIEJveGNveCAoaW5kaWNhdGVzIHRoYXQgbG9nIGlzIGRlY2VudCkKbDwtYm94Y294KG1vZGVsKQpleHBvPC1yb3VuZChsJHhbd2hpY2gubWF4KGwkeSldLDIpCgojIyBDdXJyZW50IG1vZGVsCiMjIGxvZyhMb3QuRnJvbnRhZ2UpIGN1cnJlbnRseSByZW1vdmVkIHRvIGhhdmUgbW9yZSBkYXRhIHBvaW50cyAod2FzIG5vdCBpbmNsdWRlZCB3aGVuIGxlZnQgaW4gdGhlIG1vZGVsIHdpdGggQklDKQptb2RlbC4wPWxtKGxvZyhwcmljZSkgfiBNUy5TdWJDbGFzcyArIE1TLlpvbmluZyAgKyBsb2coTG90LkFyZWEpICsgIFN0cmVldCArIEFsbGV5ICsgTG90LlNoYXBlICsgTGFuZC5Db250b3VyICsgTG90LkNvbmZpZyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSAgKyBCbGRnLlR5cGUgKyBIb3VzZS5TdHlsZSArIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgICBSb29mLlN0eWxlICsgUm9vZi5NYXRsICsgRXh0ZXJpb3IuMXN0ICsgIE1hcy5WbnIuVHlwZSArIGxvZygxK01hcy5WbnIuQXJlYSkgKyBFeHRlci5Db25kICsgRXh0ZXIuUXVhbCArICAgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBCc210LllOKyBIZWF0aW5nICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuUXUgKyBHYXJhZ2UuVHlwZSArIEdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArR2FyYWdlLlF1YWwrR2FyYWdlLkNvbmQgICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIFBvb2wuUUMgKyBGZW5jZSArIE1pc2MuVmFsICsgTW8uU29sZCAgK1lyLlNvbGQgKyBTYWxlLlR5cGUgICsgbG9nKFRvdGFsU3EpICsgUG9vbC5ZTiAsIGRhdGE9ZGF0YV90cmFpbikKCgojIFRoZXJlIGFyZSBzb21lIHBlcmZlY3QgY29sbGluZWFyaXRpZXMgaW4gdGhpcyBtb2RlbCAtPiBlbGltaW5hdGUgaXQgdmlhIEFJQy9CSUMKc3VtbWFyeShtb2RlbC4wKQpwbG90KG1vZGVsLjApCgojQUlDCiNtb2RlbC5BSUM9c3RlcChtb2RlbC4wLCBrPTIpCiNzdW1tYXJ5KG1vZGVsLkFJQykKI3Bsb3QobW9kZWwuQUlDKQoKI0JJQwojbW9kZWwuQklDPXN0ZXAobW9kZWwuMCwgaz1sb2cobnJvdyhkYXRhX3RyYWluKSkpCgoKbW9kZWwuQklDPWxtKGZvcm11bGEgPSBsb2cocHJpY2UpIH4gbG9nKExvdC5BcmVhKSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgCiAgICAgICAgICAgICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIAogICAgICAgICAgICAgICBCc210LkV4cG9zdXJlICsgbG9nKFRvdGFsLkJzbXQuU0YpICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICsgCiAgICAgICAgICAgICAgIEJhdGhzICsgQmVkcm9vbS5BYnZHciArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgCiAgICAgICAgICAgICAgIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgbG9nKFBvb2wuQXJlYSkgKyAKICAgICAgICAgICAgICAgbG9nKFRvdGFsU3EpLCBkYXRhID0gZGF0YV90cmFpbikKCgpzdW1tYXJ5KG1vZGVsLkJJQykKcGxvdChtb2RlbC5CSUMpCgojIEV4cGxvcmluZyB0aGUgcmVtYWluaW5nIHByZWRpY3RvcnMgcmVsYXRpb25zaGlwIHRvIHByaWNlCiMgcGxvdChsb2coZGF0YV90cmFpbiRMb3QuQXJlYSksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJE5laWdoYm9yaG9vZCksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJENvbmRpdGlvbi4xKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kT3ZlcmFsbC5RdWFsKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kT3ZlcmFsbC5Db25kKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kSG91c2VBZ2UpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRCc210LlF1YWwpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRCc210LkV4cG9zdXJlKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QobG9nKGRhdGFfdHJhaW4kVG90YWwuQnNtdC5TRiksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEhlYXRpbmcuUUMpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRDZW50cmFsLkFpciksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEJhdGhzKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kQmVkcm9vbS5BYnZHciksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJEtpdGNoZW4uQWJ2R3IpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRLaXRjaGVuLlF1YWwpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRGdW5jdGlvbmFsKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kRmlyZXBsYWNlcyksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KChkYXRhX3RyYWluJFBhdmVkLkRyaXZlKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QoKGRhdGFfdHJhaW4kR2FyYWdlLkNhcnMpLCBsb2coZGF0YV90cmFpbiRwcmljZSkpCiMgcGxvdCgoZGF0YV90cmFpbiRHYXJhZ2UuQ2FycyksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKIyBwbG90KGxvZygxK2RhdGFfdHJhaW4kUG9vbC5BcmVhKSwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpKQojIHBsb3QobG9nKGRhdGFfdHJhaW4kVG90YWxTcSksIGxvZyhkYXRhX3RyYWluJHByaWNlKSkKCnRlcm1wbG90KG1vZGVsLkJJQywgcGFydGlhbC5yZXNpZCA9IFRSVUUsIGNvbC5yZXMgPSAicHVycGxlIiwgY2V4ID0gMC41LAogICAgICAgICBydWcgPSBULCBzZSA9IFQsIHNtb290aCA9IHBhbmVsLnNtb290aCkKCgojIFRoZXJlIGFyZSAzIGhpZ2ggbGV2ZXJhZ2UgcG9pbnRzIC0gbWF5IHdhbnQgdG8gZXhjbHVkZSB0aGVtCmhoPC1oYXR2YWx1ZXMobW9kZWwuQklDKQppZDwtd2hpY2goaGg9PTEpCgpwbG90KGhhdHZhbHVlcyhtb2RlbC5CSUMpLCB0eXBlID0gImgiKQpgYGAKCgoKCgoKCmBgYHtyfQojIyBQcmVwYXJlIHRlc3QgZGF0YQpsb2FkKCJhbWVzX3Rlc3QuUmRhdGEiKQoKZGF0YT1hbWVzX3Rlc3QKCmRhdGEgPC0gZGF0YSAgJT4lICAKICAjZmlsdGVyKCFpcy5uYShMb3QuRnJvbnRhZ2UpKSAlPiUgCiAgbXV0YXRlKE1TLlN1YkNsYXNzPSBmYWN0b3IoTVMuU3ViQ2xhc3MpKSAlPiUKICBtdXRhdGUoQWxsZXkgPSAgZmFjdG9yKEFsbGV5LCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQWxsZXkpKSwgbGFiZWxzID0gYyhsZXZlbHMoQWxsZXkpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShIb3VzZUFnZSA9IFlyLlNvbGQtIHBtYXgoWWVhci5CdWlsdCwgWWVhci5SZW1vZC5BZGQpKSAgJT4lCiAgI2ZpbHRlcighaXMubmEoTWFzLlZuci5BcmVhKSkgICU+JSAKICBtdXRhdGUoQnNtdC5ZTiA9IDEqKCFpcy5uYShCc210LlF1YWwpKSkgJT4lCiAgbXV0YXRlKEJzbXQuUXVhbCA9ICBmYWN0b3IoQnNtdC5RdWFsLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5RdWFsKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuUXVhbCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEJzbXQuUXVhbCA9IHJlbGV2ZWwoQnNtdC5RdWFsLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXQuQ29uZCA9ICBmYWN0b3IoQnNtdC5Db25kLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuQ29uZCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEJzbXQuQ29uZCA9IHJlbGV2ZWwoQnNtdC5Db25kLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXQuRXhwb3N1cmUgPSAgZmFjdG9yKEJzbXQuRXhwb3N1cmUsIGxldmVscyA9IGxldmVscyhhZGROQShCc210LkV4cG9zdXJlKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuRXhwb3N1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210LkV4cG9zdXJlID0gcmVsZXZlbChCc210LkV4cG9zdXJlLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEJzbXRGaW4uVHlwZS4xPSAgZmFjdG9yKEJzbXRGaW4uVHlwZS4xLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdEZpbi5UeXBlLjEpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdEZpbi5UeXBlLjEpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShCc210RmluLlR5cGUuMSA9IHJlbGV2ZWwoQnNtdEZpbi5UeXBlLjEsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoQnNtdEZpbi5UeXBlLjI9ICBmYWN0b3IoQnNtdEZpbi5UeXBlLjIsIGxldmVscyA9IGxldmVscyhhZGROQShCc210RmluLlR5cGUuMikpLCBsYWJlbHMgPSBjKGxldmVscyhCc210RmluLlR5cGUuMiksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEJzbXRGaW4uVHlwZS4yID0gcmVsZXZlbChCc210RmluLlR5cGUuMiwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShYMTIuU0Y9IFgxc3QuRmxyLlNGKyBYMm5kLkZsci5TRikgICU+JSAgIAogICNmaWx0ZXIoIWlzLm5hKEJzbXQuRnVsbC5CYXRoKSkgJT4lCiAgI2ZpbHRlcighaXMubmEoQnNtdC5IYWxmLkJhdGgpKSAlPiUKICBtdXRhdGUoQmF0aHMgPSBCc210LkZ1bGwuQmF0aCArIDAuNSpCc210LkhhbGYuQmF0aCArIEZ1bGwuQmF0aCArIDAuNSpIYWxmLkJhdGgpICU+JQogIG11dGF0ZShGaXJlcGxhY2UuWU4gPSAxKihGaXJlcGxhY2VzPjApKSAlPiUKICBtdXRhdGUoRmlyZXBsYWNlLlF1ID0gIGZhY3RvcihGaXJlcGxhY2UuUXUsIGxldmVscyA9IGxldmVscyhhZGROQShGaXJlcGxhY2UuUXUpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmlyZXBsYWNlLlF1KSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoRmlyZXBsYWNlLlF1ID0gcmVsZXZlbChGaXJlcGxhY2UuUXUsIHJlZj0iTm9uZSIpKSAlPiUKICBtdXRhdGUoR2FyYWdlLllOID0gMSooIWlzLm5hKEdhcmFnZS5Db25kKSkpICU+JQogIG11dGF0ZShHYXJhZ2UuVHlwZSA9ICBmYWN0b3IoR2FyYWdlLlR5cGUsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuVHlwZSkpLCBsYWJlbHMgPSBjKGxldmVscyhHYXJhZ2UuVHlwZSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5UeXBlID0gcmVsZXZlbChHYXJhZ2UuVHlwZSwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuRmluaXNoID0gIGZhY3RvcihHYXJhZ2UuRmluaXNoLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoR2FyYWdlLkZpbmlzaCkpLCBsYWJlbHMgPSBjKGxldmVscyhHYXJhZ2UuRmluaXNoKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkZpbmlzaCA9IHJlbGV2ZWwoR2FyYWdlLkZpbmlzaCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuUXVhbCA9ICBmYWN0b3IoR2FyYWdlLlF1YWwsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuUXVhbCkpLCBsYWJlbHMgPSBjKGxldmVscyhHYXJhZ2UuUXVhbCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5RdWFsID0gcmVsZXZlbChHYXJhZ2UuUXVhbCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShHYXJhZ2UuQ29uZCA9ICBmYWN0b3IoR2FyYWdlLkNvbmQsIGxldmVscyA9IGxldmVscyhhZGROQShHYXJhZ2UuQ29uZCkpLCBsYWJlbHMgPSBjKGxldmVscyhHYXJhZ2UuQ29uZCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5Db25kID0gcmVsZXZlbChHYXJhZ2UuQ29uZCwgcmVmPSJOb25lIikpICU+JQogIG11dGF0ZShQb3JjaC5BcmVhID0gV29vZC5EZWNrLlNGKyBPcGVuLlBvcmNoLlNGK0VuY2xvc2VkLlBvcmNoK1gzU3NuLlBvcmNoICsgU2NyZWVuLlBvcmNoKSAlPiUKICBtdXRhdGUoUG9vbC5ZTiA9IDEqKFBvb2wuQXJlYT4wKSkgJT4lCiAgbXV0YXRlKFBvb2wuUUMgPSAgZmFjdG9yKFBvb2wuUUMsIGxldmVscyA9IGxldmVscyhhZGROQShQb29sLlFDKSksIGxhYmVscyA9IGMobGV2ZWxzKFBvb2wuUUMpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShQb29sLlFDID0gcmVsZXZlbChQb29sLlFDLCByZWY9Ik5vbmUiKSkgJT4lCiAgbXV0YXRlKEZlbmNlID0gIGZhY3RvcihGZW5jZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEZlbmNlKSksIGxhYmVscyA9IGMobGV2ZWxzKEZlbmNlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoTWlzYy5GZWF0dXJlID0gIGZhY3RvcihNaXNjLkZlYXR1cmUsIGxldmVscyA9IGxldmVscyhhZGROQShNaXNjLkZlYXR1cmUpKSwgbGFiZWxzID0gYyhsZXZlbHMoTWlzYy5GZWF0dXJlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoTW8uU29sZCA9IGFzLmZhY3RvcihNby5Tb2xkKSkgJT4lCiAgbXV0YXRlKFlyLlNvbGQgPSBhcy5mYWN0b3IoWXIuU29sZCkpICU+JQogIGRwbHlyOjpzZWxlY3QoLUdhcmFnZS5Zci5CbHQpICU+JQogIG11dGF0ZShDb25kaXRpb24uMSA9IGFzLmNoYXJhY3RlcihDb25kaXRpb24uMSkpICU+JQogIG11dGF0ZShLaXRjaGVuLlF1YWw9cGx5cjo6bWFwdmFsdWVzKEtpdGNoZW4uUXVhbCwgZnJvbSA9IGMoIlBvIiwgIkZhIiwgIlRBIiwiR2QiLCAiRXgiICksIHRvID0gYygiMSIsICIyIiwgIjMiLCAiNCIsICI1IikpKSAlPiUKICBtdXRhdGUoS2l0Y2hlbi5RdWFsID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIoS2l0Y2hlbi5RdWFsKSkpICU+JQogIG11dGF0ZShIZWF0aW5nLlFDPXBseXI6Om1hcHZhbHVlcyhIZWF0aW5nLlFDLCBmcm9tID0gYygiUG8iLCAiRmEiLCAiVEEiLCJHZCIsICJFeCIgKSwgdG8gPSBjKCIxIiwgIjIiLCAiMyIsICI0IiwgIjUiKSkpICU+JQogIG11dGF0ZShIZWF0aW5nLlFDID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIoSGVhdGluZy5RQykpKSAlPiUKICBtdXRhdGUoQnNtdC5RdWFsID0gZHJvcGxldmVscyhCc210LlF1YWwpKSAlPiUKICBtdXRhdGUoRnVuY3Rpb25hbCA9IGRyb3BsZXZlbHMoRnVuY3Rpb25hbCkpICU+JQogIG11dGF0ZShSb29mLk1hdGwgPSBkcm9wbGV2ZWxzKFJvb2YuTWF0bCkpCgppbmRfcmFpbDwtd2hpY2goZGF0YSRDb25kaXRpb24uMT09IlJSTm4iIHwgZGF0YSRDb25kaXRpb24uMT09IlJSQW4iIHwgZGF0YSRDb25kaXRpb24uMT09IlJSTmUiIHwgZGF0YSRDb25kaXRpb24uMT09IlJSQWUiKQppbmRfcGFyazwtd2hpY2goZGF0YSRDb25kaXRpb24uMT09IlBvc04iIHwgZGF0YSRDb25kaXRpb24uMT09IlBvc0EiKQpkYXRhJENvbmRpdGlvbi4xW2luZF9yYWlsXTwtIlJhaWwiCmRhdGEkQ29uZGl0aW9uLjFbaW5kX3BhcmtdPC0iUGFyayIKCmRhdGEgPSBkYXRhICU+JSAKICBtdXRhdGUoQ29uZGl0aW9uLjEgPSBmYWN0b3IoQ29uZGl0aW9uLjEpKSAlPiUKICBtdXRhdGUoQ29uZGl0aW9uLjEgPSByZWxldmVsKENvbmRpdGlvbi4xLCByZWY9Ik5vcm0iKSkgCgpkYXRhX3Rlc3Q9ZGF0YQoKCmRhdGFfdGVzdCRCc210LkV4cG9zdXJlW3doaWNoKGRhdGFfdGVzdCRCc210LkV4cG9zdXJlPT0iIildPC0iTm9uZSIKZGF0YV90ZXN0JEJzbXQuRXhwb3N1cmU8LWRyb3BsZXZlbHMoZGF0YV90ZXN0JEJzbXQuRXhwb3N1cmUpCgoKZGF0YV90ZXN0JFBvb2wuQXJlYTwtZGF0YV90ZXN0JFBvb2wuQXJlYSsxCmRhdGFfdGVzdCRUb3RhbC5Cc210LlNGPC1kYXRhX3Rlc3QkVG90YWwuQnNtdC5TRisxCgoKIyBleHRyYWN0IHRoZSB0cnV0aApZID0gZGF0YV90ZXN0JHByaWNlCgojIG1ha2UgcHJlZGljdGlvbiBiYXNlZCBvbiBzcGVjaWZpYyBtb2RlbApZaGF0ID0gcHJlZGljdChtb2RlbC5CSUMsIG5ld2RhdGE9ZGF0YV90ZXN0LCBpbnRlcnZhbD0icHJlZGljdCIpCgojIGRlcGVuZGluZyBvbiB0aGUgcmVzcG9uc2UgdHJhbnNmb3JtYXRpb24KWWhhdCA9IGV4cChZaGF0KQoKCiMgbmFtZSBkYXRhZnJhbWUgYXMgcHJlZGljdGlvbnMhIERPIE5PVCBDSEFOR0UKcHJlZGljdGlvbnMgPSBhcy5kYXRhLmZyYW1lKFloYXQpCnByZWRpY3Rpb25zJFBJRCA9IGRhdGFfdGVzdCRQSUQKCnNhdmUocHJlZGljdGlvbnMsIGZpbGU9InByZWRpY3QuUmRhdGEiKQoKcGVyZm9ybWFuY2UoWSwgWWhhdCkKYGBgCgoKYGBge3J9CiMgTGFzc28gZm9yIHRoZSBzaW1wbGlmaWVkIG1vZGVsIChmcm9tIEJJQykKbGlicmFyeShnbG1uZXQpCgpYLnRyYWluID0gbW9kZWwubWF0cml4KGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgICAgICAgICAgICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgQnNtdC5RdWFsICsgCiAgICAgICAgICAgICAgIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyAKICAgICAgICAgICAgICAgQmF0aHMgKyBCZWRyb29tLkFidkdyICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIEZ1bmN0aW9uYWwgKyAKICAgICAgICAgICAgICAgRmlyZXBsYWNlcyArIEdhcmFnZS5DYXJzICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIAogICAgICAgICAgICAgICBsb2coVG90YWxTcSksIGRhdGEgPSBkYXRhX3RyYWluKQoKClgudGVzdCA9IG1vZGVsLm1hdHJpeChsb2cocHJpY2UpIH4gbG9nKExvdC5BcmVhKSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgCiAgICAgICAgICAgICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIAogICAgICAgICAgICAgICBCc210LkV4cG9zdXJlICsgbG9nKFRvdGFsLkJzbXQuU0YpICsgSGVhdGluZy5RQyArIENlbnRyYWwuQWlyICsgCiAgICAgICAgICAgICAgIEJhdGhzICsgQmVkcm9vbS5BYnZHciArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgCiAgICAgICAgICAgICAgIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgbG9nKFBvb2wuQXJlYSkgKyAKICAgICAgICAgICAgICAgbG9nKFRvdGFsU3EpLCBkYXRhID0gZGF0YV90ZXN0KQoKCm1vZGVsLmxhc3NvID0gZ2xtbmV0KFgudHJhaW4sIGxvZyhkYXRhX3RyYWluJHByaWNlKSwgYWxwaGE9MSkKY3YubGFzc28gPSBjdi5nbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0xKQp5aGF0Lmxhc3NvID0gZXhwKHByZWRpY3QobW9kZWwubGFzc28sIHM9Y3YubGFzc28kbGFtYmRhLm1pbiwgdHlwZT0icmVzcG9uc2UiLCBuZXd4ID0gWC50ZXN0KSkKCnNxcnQobWVhbigoKHloYXQubGFzc28pLWRhdGFfdGVzdCRwcmljZSleMikpCgoKIyBSaWRnZSBmb3IgdGhlIHNpbXBsaWZpZWQgbW9kZWwgKGZyb20gQklDKQoKbW9kZWwucmlkZ2UgPSBnbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0wKQpjdi5yaWRnZSA9IGN2LmdsbW5ldChYLnRyYWluLCBsb2coZGF0YV90cmFpbiRwcmljZSksIGFscGhhPTApCnloYXQucmlkZ2UgPSBwcmVkaWN0KG1vZGVsLnJpZGdlLCBzPWN2LnJpZGdlJGxhbWJkYS5taW4sIHR5cGU9InJlc3BvbnNlIiwgbmV3eCA9IFgudGVzdCkKCnNxcnQobWVhbigoZXhwKHloYXQucmlkZ2UpLWRhdGFfdGVzdCRwcmljZSleMikpCmBgYAoKCgoKYGBge3J9CiMgQk1BIGZvciBzaW1wbGUgbW9kZWwgKGZyb20gQklDKQpsaWJyYXJ5KEJBUykKbW9kZWwuYm1hID0gYmFzLmxtKGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgICAgICAgICAgICAgICAgICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgQnNtdC5RdWFsICsgCiAgICAgICAgICAgICAgICAgICAgIEJzbXQuRXhwb3N1cmUgKyBsb2coVG90YWwuQnNtdC5TRikgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyAKICAgICAgICAgICAgICAgICAgICAgQmF0aHMgKyBCZWRyb29tLkFidkdyICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIEZ1bmN0aW9uYWwgKyAKICAgICAgICAgICAgICAgICAgICAgRmlyZXBsYWNlcyArIEdhcmFnZS5DYXJzICsgUGF2ZWQuRHJpdmUgKyBsb2coUG9vbC5BcmVhKSArIAogICAgICAgICAgICAgICAgICAgICBsb2coVG90YWxTcSksIGRhdGE9ZGF0YV90cmFpbiwgIHByaW9yPSJnLXByaW9yIiwgYT1ucm93KGRhdGFfdHJhaW4pLCBtb2RlbHByaW9yPXVuaWZvcm0oKSwgbWV0aG9kPSJNQ01DIiwgTUNNQy5pdGVyYXRpb25zID0gMjAwMDAwLCB0aGluID0gMjApCgp5X3ByZWQ8LXByZWRpY3QuYmFzKG1vZGVsLmJtYSxuZXdkYXRhPWRhdGFfdGVzdCwgZXN0aW1hdG9yPSJCUE0iLCBwcmVkaWN0aW9uPVRSVUUsIHNlLmZpdD1UUlVFKQoKeV9oYXQ8LWV4cChjb25maW50LnByZWQuYmFzKHlfcHJlZCkpCgp5X2hhdD0geV9oYXRbLGMoMywxLDIpXQoKcGVyZm9ybWFuY2UoZGF0YV90ZXN0JHByaWNlLCB5X2hhdCkKYGBgCgoKCgpgYGB7cn0KbGlicmFyeShnZ3Bsb3QyKQojRGF0YSBWaXp1YWxpemF0aW9ucwojIG5laWdoYm9yaG9vZC5wcmljZS5yYW5nZSA9IGdncGxvdChkYXRhX3RyYWluLCBhZXMoeCA9IGxvZyhwcmljZSksIHkgPSByZW9yZGVyKE5laWdoYm9yaG9vZCwgZGVzYyhsb2cocHJpY2UpKSwgbWVkaWFuKSkpICsKIyAgIGdlb21fcG9seWdvbihjb2xvcj0iQmxhY2siLHNpemUgPSAxLCBhbHBoYT0xKSArIGdlb21fcG9pbnQoc3RhdD0iaWRlbnRpdHkiLCBhZXMoY29sb3I9InJlZDMiKSwgc2l6ZT0xLjUpICsgdGhlbWVfbGlnaHQoKSArIGNvb3JkX2ZsaXAoKSArIHRoZW1lKGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT05MCxoanVzdD0xLHZqdXN0PTAuNSkpICsgbGFicyh5PSJOZWlnaGJvcmhvb2QiKQoKbmVpZ2hib3Job29kLnByaWNlLnJhbmdlID0gZ2dwbG90KGRhdGFfdHJhaW4sIGFlcyh5ID0gbG9nKHByaWNlKSwgeCA9IHJlb3JkZXIoTmVpZ2hib3Job29kLCBkZXNjKGxvZyhwcmljZSkpLCBtZWRpYW4pKSkgKyAKICAgICAgICAgICAgICAgICAgICAgICAgICAgZ2VvbV92aW9saW4oKSArIHRoZW1lKGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT05MCxoanVzdD0xLHZqdXN0PTAuNSkpCnBsb3QobmVpZ2hib3Job29kLnByaWNlLnJhbmdlKQpgYGAKCgoKYGBge3J9CmxpYnJhcnkodHJlZW1hcCkKCnRyZWUuZGF0YSA9IGRhdGFfdHJhaW4gJT4lIG11dGF0ZShwcmljZS5iaW4gPSBpZmVsc2UocHJpY2UgPD0gNTAwMDAsICI8PSAkNTAsMDAwIiwgaWZlbHNlKHByaWNlID4gNTAwMDAgJiBwcmljZSA8PSAxMDAwMDAsICIkNTAsMDAwLTEwMCwwMDAiLCBpZmVsc2UocHJpY2UgPiAxMDAwMDAgJiBwcmljZSA8PSAyMDAwMDAsICIkMTAwLDAwMC0yMDAsMDAwIiwgaWZlbHNlKHByaWNlID4gMjAwMDAwICYgcHJpY2UgPD0gMzAwMDAwLCAiJDIwMCwwMDAtMzAwLDAwMCIsIGlmZWxzZShwcmljZSA+IDMwMDAwMCAmIHByaWNlIDw9IDQwMDAwMCwgIiQzMDAsMDAwLTQwMCwwMDAiLCBpZmVsc2UocHJpY2UgPiA0MDAwMDAgJiBwcmljZSA8PSA1MDAwMDAsICIkNDAwLDAwMC01MDAsMDAwIiwgIj4gJDUwMCwwMDAiKSkpKSkpKQoKYXJlYS5wbG90ID0gdHJlZW1hcCh0cmVlLmRhdGEsIGluZGV4PSJwcmljZS5iaW4iLCB2U2l6ZT0iTG90LkFyZWEiLCB0eXBlPSJpbmRleCIsIHBhbGV0dGU9IllsR24iLCBhbGdvcml0aG09InBpdm90U2l6ZSIsIHRpdGxlPSJFZmZlY3Qgb2YgTG90IEFyZWEgb24gUHJpY2UgUmFuZ2UiLCBmb250c2l6ZS50aXRsZSA9IDE0KQoKZGF0YV90cmFpbiRPdmVyYWxsLlF1YWwgPSBhcy5udW1lcmljKGRhdGFfdHJhaW4kT3ZlcmFsbC5RdWFsKQpxdWFsaXR5LnBsb3QgPSBnZ3Bsb3QodHJhaW4sIGFlcyh4ID0gT3ZlcmFsbC5RdWFsLCB5PXByaWNlKSkgKyBnZW9tX3Ntb290aChjb2xvcj0iZ29sZDMiKSArIGdlb21fY291bnQoY29sb3I9InJlZDMiKSArIHRoZW1lX21pbmltYWwoKQpwbG90KHF1YWxpdHkucGxvdCkKYGBgCgoKCgoKCgoKCgoKCmBgYHtyfQpsaWJyYXJ5KE1BU1MpCgojRnVsbCBNb2RlbAptb2RlbD1sbShsb2cocHJpY2UpIH4gTVMuU3ViQ2xhc3MgKyBNUy5ab25pbmcgKyBsb2coTG90LkZyb250YWdlKSArIGxvZyhMb3QuQXJlYSkgKyAgU3RyZWV0ICsgQWxsZXkgKyBMb3QuU2hhcGUgKyBMYW5kLkNvbnRvdXIgKyBMb3QuQ29uZmlnICsgTGFuZC5TbG9wZSArIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBSb29mLlN0eWxlICsgUm9vZi5NYXRsICsgRXh0ZXJpb3IuMXN0ICsgRXh0ZXJpb3IuMm5kICsgTWFzLlZuci5UeXBlICsgbG9nKDErTWFzLlZuci5BcmVhKSArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXQuWU46QnNtdC5RdWFsICsgQnNtdC5ZTjpCc210LkNvbmQgKyBCc210LllOOkJzbXQuRXhwb3N1cmUgKyBUb3RhbC5Cc210LlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIEVsZWN0cmljYWwgKyBsb2coWDEyLlNGKSArIGxvZygxK0xvdy5RdWFsLkZpbi5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgS2l0Y2hlbi5RdWFsICsgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBGaXJlcGxhY2UuWU46RmlyZXBsYWNlLlF1ICsgR2FyYWdlLllOOkdhcmFnZS5UeXBlICsgR2FyYWdlLllOOkdhcmFnZS5GaW5pc2ggKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5ZTjpHYXJhZ2UuQ29uZCArIEdhcmFnZS5ZTjpHYXJhZ2UuUXVhbCArIFBhdmVkLkRyaXZlICsgbG9nKDErUG9vbC5BcmVhKSArIFBvb2wuWU46UG9vbC5RQyArIEZlbmNlICsgTWlzYy5WYWwgKyBNby5Tb2xkICsgWXIuU29sZCArIFNhbGUuVHlwZSAgKyBUb3RhbFNxLCBkYXRhPXRyYWluKQoKc3VtbWFyeShtb2RlbCkKYm94Y294KG1vZGVsKQojQklDIG1vZGVsIHNlbGVjdGlvbgpzdGVwKG1vZGVsLCBrPWxvZyhucm93KHRyYWluKSkpCgojc3RlcChtb2RlbCwgaz0yKQojIE9MRAojIG1vZGVsPWxtKGxvZyhwcmljZSl+bXMuc3ViY2xhc3MgKyBNUy5ab25pbmcgKyBsb2coTG90LkZyb250YWdlKSArIGxvZyhhcmVhKSArIFN0cmVldCArIEFsbGV5ICsgTG90LlNoYXBlICsgTGFuZC5Db250b3VyICsgTG90LkNvbmZpZyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgUm9vZi5TdHlsZSArIEV4dGVyaW9yLjFzdCArIEV4dGVyaW9yLjJuZCArIE1hcy5WbnIuVHlwZSArIGxvZyhNYXMuVm5yLkFyZWEgKyAxKSArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXQuUXVhbCArIEJzbXQuQ29uZCArIEJzbXQuRXhwb3N1cmUgKyBIZWF0aW5nLlFDICsgQ2VudHJhbC5BaXIgKyBFbGVjdHJpY2FsICsgbG9nKFgxc3QuRmxyLlNGKSArIGxvZyhYMm5kLkZsci5TRikgKyBCYXRocyArIEJlZHJvb20uQWJ2R3IgKyBLaXRjaGVuLkFidkdyICsgVG90Um1zLkFidkdyZCArIFBhdmVkLkRyaXZlICsgbG9nKFdvb2QuRGVjay5TRiArIDEpICsgbG9nKE9wZW4uUG9yY2guU0YgKyAxKSArIGxvZyhFbmNsb3NlZC5Qb3JjaCArIDEpICsgbG9nKFgzU3NuLlBvcmNoICsgMSkgKyBsb2coU2NyZWVuLlBvcmNoICsgMSkgKyBsb2coTWlzYy5WYWwgKyAxKSArIE1vLlNvbGQgKyBZci5Tb2xkICsgU2FsZS5UeXBlICsgbG9nKFRvdGFsU3EpLCBkYXRhPXRyYWluKQojdmlmKG1vZGVsKQojc3VtbWFyeShtb2RlbCkKCiNNb2RlbCBTZWxlY3Rpb24gYnkgQUlDCiNzdGVwKG1vZGVsLCBrPTIpCiMjQUlDX21vZGVsID0gbG0oZm9ybXVsYSA9IGxvZyhwcmljZSkgfiBQSUQgKyBhcmVhICsgTVMuU3ViQ2xhc3MgKyBNUy5ab25pbmcgKyAKIyAgICBMb3QuQXJlYSArIExvdC5TaGFwZSArIFV0aWxpdGllcyArIExhbmQuU2xvcGUgKyBOZWlnaGJvcmhvb2QgKyAKIyAgICBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIEhvdXNlLlN0eWxlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICAgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgUm9vZi5TdHlsZSArIFJvb2YuTWF0bCArIEV4dGVyaW9yLjFzdCArIAojICAgIEV4dGVyaW9yLjJuZCArIEV4dGVyLlF1YWwgKyBFeHRlci5Db25kICsgRm91bmRhdGlvbiArIEJzbXRGaW4uU0YuMSArIAojICAgIEJzbXRGaW4uU0YuMiArIEJzbXQuVW5mLlNGICsgSGVhdGluZyArIEhlYXRpbmcuUUMgKyBDZW50cmFsLkFpciArIAojICAgIEVsZWN0cmljYWwgKyBYMXN0LkZsci5TRiArIFgybmQuRmxyLlNGICsgQnNtdC5GdWxsLkJhdGggKyAKIyAgICBLaXRjaGVuLlF1YWwgKyBGdW5jdGlvbmFsICsgR2FyYWdlLkNhcnMgKyBHYXJhZ2UuQXJlYSArIFBhdmVkLkRyaXZlICsgCiMgICAgV29vZC5EZWNrLlNGICsgT3Blbi5Qb3JjaC5TRiArIEVuY2xvc2VkLlBvcmNoICsgU2NyZWVuLlBvcmNoICsgCiMgICAgTW8uU29sZCArIFlyLlNvbGQgKyBoYXMuZmVuY2UgKyBoYXMuZmlyZXBsYWNlLCBkYXRhID0gdHJhaW4pCiNzdW1tYXJ5KEFJQ19tb2RlbCkKCiNNb2RlbCBTZWxlY3Rpb24gYnkgQklDCiNCSUNfbW9kZWwgPSBsbShmb3JtdWxhID0gbG9nKHByaWNlKSB+IFBJRCArIGFyZWEgKyBNUy5ab25pbmcgKyBMb3QuQXJlYSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgQmxkZy5UeXBlICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICBZZWFyLkJ1aWx0ICsgWWVhci5SZW1vZC5BZGQgKyBCc210RmluLlNGLjEgKyBCc210RmluLlNGLjIgKyAKIyAgICBCc210LlVuZi5TRiArIENlbnRyYWwuQWlyICsgQnNtdC5GdWxsLkJhdGggKyBLaXRjaGVuLlF1YWwgKyAKIyAgICBGdW5jdGlvbmFsICsgR2FyYWdlLkNhcnMgKyBHYXJhZ2UuQXJlYSArIE9wZW4uUG9yY2guU0YgKyAKIyAgICBFbmNsb3NlZC5Qb3JjaCArIFNjcmVlbi5Qb3JjaCArIGhhcy5maXJlcGxhY2UsIGRhdGEgPSB0cmFpbikKI3N1bW1hcnkoQklDX21vZGVsKQoKI1VzZSBCb29zdGluZyB0byBmaW5kIGltcG9ydGFudCB2YXJpYWJsZXMKIwojbGlicmFyeShnYm0pCiMKI2Jvb3N0ID0gZ2JtKGxvZyhwcmljZSkgfiBQSUQgKyBhcmVhICsgTVMuWm9uaW5nICsgTG90LkFyZWEgKyAKIyAgICBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIEJsZGcuVHlwZSArIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIAojICAgIFllYXIuQnVpbHQgKyBZZWFyLlJlbW9kLkFkZCArIEJzbXRGaW4uU0YuMSArIEJzbXRGaW4uU0YuMiArIAojICAgIEJzbXQuVW5mLlNGICsgQ2VudHJhbC5BaXIgKyBCc210LkZ1bGwuQmF0aCArIEtpdGNoZW4uUXVhbCArIAojICAgIEZ1bmN0aW9uYWwgKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5BcmVhICsgT3Blbi5Qb3JjaC5TRiArIAojICAgIEVuY2xvc2VkLlBvcmNoICsgU2NyZWVuLlBvcmNoICsgaGFzLmZpcmVwbGFjZSwgZGF0YSA9IHRyYWluLCBkaXN0cmlidXRpb249ImdhdXNzaWFuIiwgbi50cmVlcz01MDAwLCAjaW50ZXJhY3Rpb24uZGVwdGggPSAxLCBzaHJpbmthZ2U9MC4wMSwgdmVyYm9zZSA9IEYpCgojc3VtbWFyeShib29zdCkKCiMjU3BhcnNlIE1vZGVsIChyZW1vdmVkIHZhcmlhYmxlcyB0byBicmluZyB0aGUgbnVtYmVyIGJlbG93IDIwKQojc3BhcnNlX21vZGVsID0gbG0obG9nKHByaWNlKSB+IGFyZWEgKyBNUy5ab25pbmcgKyBMb3QuQXJlYSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgCiMgICAgWWVhci5CdWlsdCArIFllYXIuUmVtb2QuQWRkICsgQnNtdEZpbi5TRi4xICsgQnNtdEZpbi5TRi4yICsgCiMgICAgQnNtdC5VbmYuU0YgKyBDZW50cmFsLkFpciArIEJzbXQuRnVsbC5CYXRoICsgS2l0Y2hlbi5RdWFsICsgCiMgICAgRnVuY3Rpb25hbCArIEdhcmFnZS5DYXJzICsgR2FyYWdlLkFyZWEgKyBoYXMuZmlyZXBsYWNlLCBkYXRhID0gdHJhaW4pCiNzdW1tYXJ5KHNwYXJzZV9tb2RlbCkKI3Bsb3Qoc3BhcnNlX21vZGVsKQojdGVybXBsb3Qoc3BhcnNlX21vZGVsLCAKIyAgICBkYXRhID0gdHJhaW4pCgojI1RyeWluZyB0byBkZXRlY3Qgbm9uLWxpbmVhcml0eQojYXR0YWNoKHRyYWluKQojcGxvdChsb2coYXJlYSksIGxvZyhwcmljZSkpCiNwbG90KGxvZyhMb3QuQXJlYSksIGxvZyhwcmljZSkpCiNwbG90KGxvZyhPdmVyYWxsLlF1YWwpLCBsb2cocHJpY2UpKQojcGxvdChsb2coT3ZlcmFsbC5Db25kKSwgbG9nKHByaWNlKSkKI3Bsb3QobG9nKFllYXIuQnVpbHQpLCBsb2cocHJpY2UpKQojcGxvdChsb2coWWVhci5SZW1vZC5BZGQsIGxvZyhwcmljZSkpKQojcGxvdChCc210RmluLlNGLjEsIGxvZyhwcmljZSkpCiNwbG90KEJzbXRGaW4uU0YuMiwgbG9nKHByaWNlKSkKI3Bsb3QoQnNtdC5VbmYuU0YsIGxvZyhwcmljZSkpCiNwbG90KEJzbXQuRnVsbC5CYXRoLCBsb2cocHJpY2UpKQojcGxvdChHYXJhZ2UuQ2FycywgbG9nKHByaWNlKSkKI3Bsb3QoR2FyYWdlLkFyZWEsIGxvZyhwcmljZSkpCiMKIyNPTEQgU2ltcGxlIG1vZGVsCiNtb2RlbDEgPSBsbShsb2cocHJpY2UpIH4gbG9nKGFyZWEpICsgTVMuWm9uaW5nICsgbG9nKExvdC5BcmVhKSArIAojICAgIE5laWdoYm9yaG9vZCArIENvbmRpdGlvbi4xICsgbG9nKE92ZXJhbGwuUXVhbCkgKyBsb2coT3ZlcmFsbC5Db25kKSArIAojICAgIGxvZyhZZWFyLkJ1aWx0KSArIGxvZyhZZWFyLlJlbW9kLkFkZCkgKyBCc210RmluLlNGLjEgKyBCc210RmluLlNGLjIgKyAKIyAgICBCc210LlVuZi5TRiArIENlbnRyYWwuQWlyICsgQnNtdC5GdWxsLkJhdGggKwojICAgIEZ1bmN0aW9uYWwgKyBHYXJhZ2UuQ2FycyArIEdhcmFnZS5BcmVhICsgaGFzLmZpcmVwbGFjZSwgZGF0YSA9IHRyYWluWy1jKDE2OCwgNDYxLCA3ODcpLF0pCiNzdW1tYXJ5KG1vZGVsMSkKI3Bsb3QobW9kZWwxKQoKI1NpbXBsZSBtb2RlbCAoYmFzZWQgb24gQklDIHNlbGVjdGlvbikKbW9kZWwxID0gbG0oZm9ybXVsYSA9IGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIFRvdGFsLkJzbXQuU0YgKyAKICAgIENlbnRyYWwuQWlyICsgbG9nKFgxMi5TRikgKyBCYXRocyArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyAKICAgIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSArIEJzbXQuWU46QnNtdC5FeHBvc3VyZSwgCiAgICBkYXRhID0gdHJhaW4pCgojUGxvdCBQcmljZSBSYW5nZXMgb2YgZWFjaCBuZWlnaGJvcmhvb2QKbGlicmFyeShnZ3Bsb3QyKQoKbmVpZ2hib3Job29kLnByaWNlLnJhbmdlID0gZ2dwbG90KHRyYWluLCBhZXMoeCA9IGxvZyhwcmljZSksIHkgPSByZW9yZGVyKE5laWdoYm9yaG9vZCwgZGVzYyhsb2cocHJpY2UpKSwgbWVkaWFuKSkpICsKICBnZW9tX3BvbHlnb24oY29sb3I9IlNsYXRlR3JheSIsc2l6ZSA9IDEsIGFscGhhPTEpICsgdGhlbWVfbGlnaHQoKSArIGNvb3JkX2ZsaXAoKSArIHRoZW1lKGF4aXMudGV4dC54PWVsZW1lbnRfdGV4dChhbmdsZT05MCxoanVzdD0xLHZqdXN0PTAuNSkpICsgbGFicyh5PSJOZWlnaGJvcmhvb2QiKQoKcGxvdChuZWlnaGJvcmhvb2QucHJpY2UucmFuZ2UpCmBgYAoKCgojIyMgTW9kZWwgRXZhbHVhdGlvbiBvbiBUZXN0IERhdGEKQ3JlYXRlIHByZWRpY3RlZCB2YWx1ZXMgZm9yIHByaWNlIHVzaW5nIHlvdXIgbW9kZWwgdXNpbmcgdGhlIHRlc3RpbmcgZGF0YQoKYGBge3IgcmVhZC10ZXN0LWRhdGF9CmxvYWQoImFtZXNfdGVzdC5SZGF0YSIpCgojVmFyaWFibGVzIHdpdGggTkEncyBhbmQgdGhlaXIgcHJvcG9ydGlvbiBvZiBtaXNzaW5nIGRhdGEKbWlzcyA9IGFwcGx5KGlzLm5hKGFtZXNfdHJhaW4pLCAyLCBzdW0pCm1pc3NfcHJvcCA9IHJvdW5kKG1pc3NbbWlzcz4wXS9ucm93KGFtZXNfdHJhaW4pLCAzKQpwcmludChtaXNzX3Byb3ApIAp3aGljaChtaXNzX3Byb3A+MC41KSAjIGZvdXIgZmVhdHVyZXMgaGF2ZSBncmVhdGVyIHRoYW4gNTAlIG9mIGRhdGEgbWlzc2luZwoKCiNDcmVhdGVkIGJpbmFyeSB2YXJpYWJsZXMgZm9yIHdoZXRoZXIgb3Igbm90IGEgaG91c2UgYXMgYW4gYWxsZXksIHBvb2wsIGZlbmNlLCBtaXNjLiBmZWF0dXJlLCBmaXJlcGxhY2UsIGJhc2VtZW50IG9yIGdhcmFnZSwgYXMgSSB0aG91Z2h0IGl0IG1heSBiZSBtb3JlIG1lYW5pbmdmdWwtIFRvbQojTWF5IGNvbnNpZGVyIGFkZGluZyBMb3QuRnJvbnRhZ2UgYmFjayBpbiwgYXMgaXQgb25seSBoYWQgMjg2IG1pc3NpbmcKI0hhdmUgdG8gZGVjaWRlIGJldHdlZW4gZmlsdGVyaW5nIG91dCBOL0EncyBmb3IgZGlmZmVyZW50IEdhcmFnZSB2YXJpYWJsZXMgYW5kIG9ubHkgaW5jbHVkaW5nICdoYXMuZ2FyYWdlJwojSGF2ZSB0byBkZWNpZGUgYmV0d2VlbiBmaWx0ZXJpbmcgb3V0IE4vQSdzIGZvciBkaWZmZXJlbnQgQmFzZW1lbnQgdmFyaWFibGVzIGFuZCBvbmx5IGluY2x1ZGluZyAnaGFzLmJhc2VtZW50Jwp0ZXN0IDwtIGFtZXNfdGVzdCAgJT4lICAgZmlsdGVyKCFpcy5uYShMb3QuRnJvbnRhZ2UpKSAlPiUgCiAgbXV0YXRlKEFsbGV5ID0gIGZhY3RvcihBbGxleSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEFsbGV5KSksIGxhYmVscyA9IGMobGV2ZWxzKEFsbGV5KSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoSG91c2VBZ2UgPSBZci5Tb2xkLSBwbWF4KFllYXIuQnVpbHQsIFllYXIuUmVtb2QuQWRkKSkgICU+JQogIGZpbHRlcighaXMubmEoTWFzLlZuci5BcmVhKSkgICU+JSAKICBtdXRhdGUoQnNtdC5ZTiA9ICFpcy5uYShCc210LlF1YWwpKSAlPiUKICAgIG11dGF0ZShCc210LlF1YWwgPSAgZmFjdG9yKEJzbXQuUXVhbCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuUXVhbCkpLCBsYWJlbHMgPSBjKGxldmVscyhCc210LlF1YWwpLCAiTm9CYSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogICAgbXV0YXRlKEJzbXQuQ29uZCA9ICBmYWN0b3IoQnNtdC5Db25kLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoQnNtdC5Db25kKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXQuQ29uZCksICJOb0JhIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoQnNtdC5FeHBvc3VyZSA9ICBmYWN0b3IoQnNtdC5FeHBvc3VyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXQuRXhwb3N1cmUpKSwgbGFiZWxzID0gYyhsZXZlbHMoQnNtdC5FeHBvc3VyZSksICJOb0JhIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgIG11dGF0ZShCc210RmluLlR5cGUuMT0gIGZhY3RvcihCc210RmluLlR5cGUuMSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4xKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4xKSwgIk5vQmEiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICAgIG11dGF0ZShCc210RmluLlR5cGUuMj0gIGZhY3RvcihCc210RmluLlR5cGUuMiwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEJzbXRGaW4uVHlwZS4yKSksIGxhYmVscyA9IGMobGV2ZWxzKEJzbXRGaW4uVHlwZS4yKSwgIk5vQmEiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoWDEyLlNGPSBYMXN0LkZsci5TRisgWDJuZC5GbHIuU0YpICAlPiUgICAKICBmaWx0ZXIoIWlzLm5hKEJzbXQuRnVsbC5CYXRoKSkgJT4lCiAgZmlsdGVyKCFpcy5uYShCc210LkhhbGYuQmF0aCkpICU+JQogIG11dGF0ZShCYXRocyA9IEJzbXQuRnVsbC5CYXRoICsgMC41KkJzbXQuSGFsZi5CYXRoICsgRnVsbC5CYXRoICsgMC41KkhhbGYuQmF0aCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5ZTiA9IEZpcmVwbGFjZXM+MCkgJT4lCiAgbXV0YXRlKEZpcmVwbGFjZS5RdSA9ICBmYWN0b3IoRmlyZXBsYWNlLlF1LCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmlyZXBsYWNlLlF1KSksIGxhYmVscyA9IGMobGV2ZWxzKEZpcmVwbGFjZS5RdSksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgbXV0YXRlKEdhcmFnZS5ZTiA9ICFpcy5uYShHYXJhZ2UuQ29uZCkpICU+JQogICAgbXV0YXRlKEdhcmFnZS5UeXBlID0gIGZhY3RvcihHYXJhZ2UuVHlwZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5UeXBlKSksIGxhYmVscyA9IGMobGV2ZWxzKEdhcmFnZS5UeXBlKSwgIk5vbmUiKSwgZXhjbHVkZSA9IE5VTEwpKSAlPiUKICBtdXRhdGUoR2FyYWdlLkZpbmlzaCA9ICBmYWN0b3IoR2FyYWdlLkZpbmlzaCwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKEdhcmFnZS5GaW5pc2gpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLkZpbmlzaCksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoR2FyYWdlLlF1YWwgPSAgZmFjdG9yKEdhcmFnZS5RdWFsLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoR2FyYWdlLlF1YWwpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLlF1YWwpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogICAgICBtdXRhdGUoR2FyYWdlLkNvbmQgPSAgZmFjdG9yKEdhcmFnZS5Db25kLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoR2FyYWdlLkNvbmQpKSwgbGFiZWxzID0gYyhsZXZlbHMoR2FyYWdlLkNvbmQpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShQb3JjaC5BcmVhID0gV29vZC5EZWNrLlNGKyBPcGVuLlBvcmNoLlNGK0VuY2xvc2VkLlBvcmNoK1gzU3NuLlBvcmNoICsgU2NyZWVuLlBvcmNoKSAlPiUKICBtdXRhdGUoUG9vbC5ZTiA9IFBvb2wuQXJlYT4wKSAlPiUKICBtdXRhdGUoUG9vbC5RQyA9ICBmYWN0b3IoUG9vbC5RQywgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKFBvb2wuUUMpKSwgbGFiZWxzID0gYyhsZXZlbHMoUG9vbC5RQyksICJOb25lIiksIGV4Y2x1ZGUgPSBOVUxMKSkgJT4lCiAgICBtdXRhdGUoRmVuY2UgPSAgZmFjdG9yKEZlbmNlLCBsZXZlbHMgPSBsZXZlbHMoYWRkTkEoRmVuY2UpKSwgbGFiZWxzID0gYyhsZXZlbHMoRmVuY2UpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNaXNjLkZlYXR1cmUgPSAgZmFjdG9yKE1pc2MuRmVhdHVyZSwgbGV2ZWxzID0gbGV2ZWxzKGFkZE5BKE1pc2MuRmVhdHVyZSkpLCBsYWJlbHMgPSBjKGxldmVscyhNaXNjLkZlYXR1cmUpLCAiTm9uZSIpLCBleGNsdWRlID0gTlVMTCkpICU+JQogIG11dGF0ZShNby5Tb2xkID0gYXMuZmFjdG9yKE1vLlNvbGQpKSAlPiUKICAgIG11dGF0ZShZci5Tb2xkID0gYXMuZmFjdG9yKFlyLlNvbGQpKSAlPiUKICBkcGx5cjo6c2VsZWN0KC1HYXJhZ2UuWXIuQmx0KSAlPiUgZmlsdGVyKENvbmRpdGlvbi4xICE9ICJSUk5lIikgJT4lIGZpbHRlcihLaXRjaGVuLlF1YWwgIT0gIlBvIikKICAgICAgICAgICAgICAgICAgCmBgYAoKYGBge3IgcHJlZGljdC1tb2RlbDEsIGVjaG89RkFMU0V9ClloYXQgPSBwcmVkaWN0KG1vZGVsMSwgbmV3ZGF0YT10ZXN0LCBpbnRlcnZhbD0icHJlZGljdGlvbiIpCmBgYAoKWW91IHNob3VsZCBzYXZlIHlvdXIgcHJlZGljdGlvbnMgaW4gYSBkYXRhZnJhbWUgd2l0aCBjb2x1bW5zIGZvciBgUElEYCAgKHByb3BlcnR5IGlkZW50aWZpZXIpLCBgZml0YCwgcHJlZGljdGVkIHZhbHVlcyBvbiB0aGUgdGVzdCBkYXRhLCBhbmQgd2hlcmUgcG9zc2libGUgYGx3cmAgYW5kIGB1cHJgLCBsb3dlciBhbmQgdXBwZXIgOTUlIGludGVydmFsIGVzdGltYXRlcyBmb3IgcHJlZGljdGluZyBgcHJpY2VgLiAKCmBgYHtyIGNyZWF0ZSB9ClkgPSB0ZXN0JHByaWNlClloYXQgPSBleHAoWWhhdCkKCiNCaWFzCm1lYW4oWWhhdFssMV0gLSBZKQoKI01heGltdW0gRGV2aWF0aW9uCm1heChhYnMoWSAtIFloYXRbLDFdKSkKCiNNZWFuIEFic29sdXRlIERldmlhdGlvbgptZWFuKGFicyhZIC0gWWhhdFssMV0pKQoKI1JNU0UKc3FydChtZWFuKChZIC0gWWhhdFssMV0pXjIpKQoKI0NvdmVyYWdlCm1lYW4oWWhhdFssImx3ciJdIDwgWSAmIFloYXRbLCJ1cHIiXSA+IFkpCgojIG5hbWUgZGF0YWZyYW1lIGFzIHByZWRpY3Rpb25zISBETyBOT1QgQ0hBTkdFCnByZWRpY3Rpb25zID0gYXMuZGF0YS5mcmFtZShZaGF0KQpwcmVkaWN0aW9ucyRQSUQgPSB0ZXN0JFBJRApzYXZlKHByZWRpY3Rpb25zLCBmaWxlPSJwcmVkaWN0LlJkYXRhIikKYGBgCgpZb3VyIG1vZGVscyB3aWxsIGJlIGV2YWx1YXRlZCBvbiB0aGUgZm9sbG93aW5nIGNyaXRlcmlhIG9uIHRoZSB0ZXN0IGRhdGE6IAoKKu+CtyBCaWFzOiAgQXZlcmFnZSAoWWhhdC1ZKSAgcG9zaXRpdmUgdmFsdWVzIGluZGljYXRlIHRoZSBtb2RlbCB0ZW5kcyB0byBvdmVyZXN0aW1hdGUgcHJpY2UgKG9uIGF2ZXJhZ2UpIHdoaWxlIG5lZ2F0aXZlIHZhbHVlcyBpbmRpY2F0ZSB0aGUgbW9kZWwgdGVuZHMgdG8gdW5kZXJlc3RpbWF0ZSBwcmljZS4KCirvgrcgTWF4aW11bSBEZXZpYXRpb246ICBNYXggfFktWWhhdHwgLSAgaWRlbnRpZmllcyB0aGUgd29yc3QgcHJlZGljdGlvbiAgbWFkZSBpbiB0aGUgdmFsaWRhdGlvbiBkYXRhIHNldC4KCirvgrcgTWVhbiBBYnNvbHV0ZSBEZXZpYXRpb246ICBBdmVyYWdlIHxZLVloYXR8IC0gdGhlIGF2ZXJhZ2UgZXJyb3IgKHJlZ2FyZGxlc3Mgb2Ygc2lnbikuCgoqIFJvb3TvgrcgTWVhbiBTcXVhcmUgRXJyb3I6IFNxcnQgQXZlcmFnZSAoWS1ZaGF0KV4yCgoqIENvdmVyYWdlOiAgQXZlcmFnZSggbHdyIDwgWSA8IHVwcikgCgpJbiBvcmRlciB0byBoYXZlIGEgcGFzc2luZyB3ZXJja2VyIGJhZGdlLCB5b3VyIGZpbGUgZm9yIHByZWRpY3Rpb25zIG5lZWRzIHRvIGJlIHRoZSBzYW1lIGxlbmd0aCBhcyB0aGUgdGVzdCBkYXRhLCB3aXRoIHRocmVlIGNvbHVtbnM6ICBmaXR0ZWQgdmFsdWVzLCBsb3dlciBDSSBhbmQgdXBwZXIgQ0kgdmFsdWVzIGluIHRoYXQgb3JkZXIgd2l0aCBuYW1lcywgZml0LCBsd3IsIGFuZCB1cHIgcmVzcGVjdGl2ZWx5LgoKWW91IHdpbGwgYmUgYWJsZSB0byBzZWUgeW91ciBzY29yZXMgb24gdGhlIHNjb3JlIGJvYXJkIChjb21pbmcgc29vbiEpLiAgVGhleSB3aWxsIGJlIGluaXRpYWxpemVkIGJ5IGEgcHJlZGN0aW9uIGJhc2VkIG9uIHRoZSBtZWFuIGluIHRoZSB0cmFpbmluZyBkYXRhLgoKX01vZGVsIENoZWNrXyAtIFRlc3QgeW91ciBwcmVkaWN0aW9uIG9uIHRoZSBmaXJzdCBvYnNlcnZhdGlvbiBpbiB0aGUgdHJhaW5pbmcgYW5kIHRlc3QgZGF0YSBzZXQgdG8gbWFrZSBzdXJlIHRoYXQgdGhlIG1vZGVsIGdpdmVzIGEgcmVhc29uYWJsZSBhbnN3ZXIgYW5kIGluY2x1ZGUgdGhpcyBpbiBhIHN1cHBsZW1lbnQgb2YgeW91ciByZXBvcnQuIFRoaXMgc2hvdWxkIGJlIGRvbmUgQlkgSEFORCB1c2luZyBhIGNhbGN1bGF0b3IgKHRoaXMgbWVhbnMgdXNlIHRoZSByYXcgZGF0YSBmcm9tIHRoZSBvcmlnaW5hbCBkYXRhc2V0IGFuZCBtYW51YWxseSBjYWxjdWxhdGUgYWxsIHRyYW5zZm9ybWF0aW9ucyBhbmQgaW50ZXJhY3Rpb25zIHdpdGggeW91ciBjYWxjdWxhdG9yKSEgTW9kZWxzIHRoYXQgZG8gbm90IGdpdmUgcmVhc29uYWJsZSBhbnN3ZXJzIHdpbGwgYmUgZ2l2ZW4gYSBtaW5pbXVtIDIgbGV0dGVyIGdyYWRlIHJlZHVjdGlvbi4gQWxzbyBiZSBjYXJlZnVsIGFzIHlvdSBjYW5ub3QgdXNlIGNlcnRhaW4gdHJhbnNmb3JtYXRpb25zIFtsb2cgb3IgaW52ZXJzZSB4XSBpZiBhIHZhcmlhYmxlIGhhcyB2YWx1ZXMgb2YgMC4KCgoKCiMjIyBQYXJ0IElJOiBDb21wbGV4IE1vZGVsCgpJbiB0aGlzIHBhcnQgeW91IG1heSBnbyBhbGwgb3V0IGZvciBjb25zdHJ1Y3RpbmcgYSBiZXN0IGZpdHRpbmcgbW9kZWwgZm9yIHByZWRpY3RpbmcgaG91c2luZyBwcmljZXMgdXNpbmcgbWV0aG9kcyB0aGF0IHdlIGhhdmUgY292ZXJlZCB0aGlzIHNlbWVzdGVyLiAgWW91IHNob3VsZCBmZWVsIGZyZWUgdG8gdG8gY3JlYXRlIGFueSBuZXcgdmFyaWFibGVzIChzdWNoIGFzIHF1YWRyYXRpYywgaW50ZXJhY3Rpb24sIG9yIGluZGljYXRvciB2YXJpYWJsZXMsIHNwbGluZXMsIGV0YykuIFRoZSB2YXJpYWJsZSBgVG90YWxTcSA9IFgxc3QuRmxyLlNGK1gybmQuRmxyLlNGYCB3YXMgYWRkZWQgdG8gdGhlIGRhdGFmcmFtZSAodGhhdCBkb2VzIG5vdCBpbmNsdWRlIGJhc2VtZW50IGFyZWEsIHNvIHlvdSBtYXkgaW1wcm92ZSBvbiB0aGlzLiBBIHJlbGF0aXZlIGdyYWRlIGlzIGFzc2lnbmVkIGJ5IGNvbXBhcmluZyB5b3VyIGZpdCBvbiB0aGUgdGVzdCBzZXQgdG8gdGhhdCBvZiB5b3VyIGZlbGxvdyBzdHVkZW50cyB3aXRoIGJvbnVzIHBvaW50cyBhd2FyZGVkIHRvIHRob3NlIHdobyBzdWJzdGFudGlhbGx5IGV4Y2VlZCB0aGVpciBmZWxsb3cgc3R1ZGVudHMgYW5kIHBvaW50IHJlZHVjdGlvbnMgb2NjdXJyaW5nIGZvciBtb2RlbHMgd2hpY2ggZml0IGV4Y2VlZGluZ2x5IHBvb3JseS4gIAoKYGBge3J9CmxpYnJhcnkoZ2xtbmV0KQojRGVzaWduIG1hdHJpY2VzClgudHJhaW4gPSBtb2RlbC5tYXRyaXgobG9nKHByaWNlKSB+IGxvZyhMb3QuQXJlYSkgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIAogICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgVG90YWwuQnNtdC5TRiArIAogICAgQ2VudHJhbC5BaXIgKyBsb2coWDEyLlNGKSArIEJhdGhzICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIAogICAgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlICsgQnNtdC5ZTjpCc210LkV4cG9zdXJlLCAKICAgIGRhdGEgPSBkYXRhKQoKWC50ZXN0ID0gbW9kZWwubWF0cml4KGxvZyhwcmljZSkgfiBsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIFRvdGFsLkJzbXQuU0YgKyAKICAgIENlbnRyYWwuQWlyICsgbG9nKFgxMi5TRikgKyBCYXRocyArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyAKICAgIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSArIEJzbXQuWU46QnNtdC5FeHBvc3VyZSwgCiAgICBkYXRhID0gdGVzdCkKCiNGaXQgbGFzc28gcmVncmVzc2lvbgpwcmljZS5sYXNzbyA9IGdsbW5ldChYLnRyYWluLCBsb2coZGF0YSRwcmljZSksIGFscGhhPTEpCgpjdi5sYXNzbyA9IGN2LmdsbW5ldChYLnRyYWluLCBsb2coZGF0YSRwcmljZSksIGFscGhhPTEpCgojT2J0YWluIHByZWRpY3Rpb24gb24gdGVzdCBkYXRhCnloYXQubGFzc28udGVzdCA9IHByZWRpY3QoYXBwLmxhc3NvLCBzPWN2Lmxhc3NvJGxhbWJkYS5taW4sIHR5cGU9InJlc3BvbnNlIiwgbmV3eCA9IFgudGVzdCkKCiNDb21wdXRlIFJNU0UgZm9yIHRlc3QgZGF0YQpybXNlLmxhc3NvLnRlc3QgPSBybXNlKHRlc3QkcHJpY2UsIGV4cCh5aGF0Lmxhc3NvLnRlc3QpKQpybXNlLnRhYiRsYXNzbzwtcm1zZS5sYXNzby50ZXN0CmBgYAoKVXBkYXRlIHlvdXIgcHJlZGljdGlvbnMgdXNpbmcgeW91ciBjb21wbGV4IG1vZGVsIHRvIHByb3ZpZGUgcG9pbnQgZXN0aW1hdGVzIGFuZCBDSS4KCmBgYHtyIHByZWRpY3QtbW9kZWwyLCBlY2hvPUZBTFNFfQojIHJlcGxhY2UgbW9kZWwxIHdpdGggbW9kZWwyCnByZWRpY3Rpb25zID0gYXMuZGF0YS5mcmFtZShwcmVkaWN0KG1vZGVsMSwgbmV3ZGF0YT1hbWVzX3Rlc3QsIGludGVydmFsID0gInByZWQiKSkKcHJlZGljdGlvbnMkUElEID0gYW1lc190ZXN0JFBJRApzYXZlKHByZWRpY3Rpb25zLCBmaWxlPSJwcmVkaWN0LlJkYXRhIikKYGBgCgpZb3UgbWF5IGl0ZXJhdGUgaGVyZSBhcyBtdWNoIGFzIHlvdSBsaWtlIGV4cGxvcmluZyBkaWZmZXJlbnQgbW9kZWxzIHVudGlsIHlvdSBhcmUgc2F0aXNmaWVkIHdpdGggeW91ciByZXN1bHRzLgoKCgoKQmF5ZXNpYW4gbGFzc28KYGBge3J9CmxpYnJhcnkoZ2xtbmV0KQpsaWJyYXJ5KG1vbm9tdm4pCgpob3VzaW5nLnJpZGdlID0gbG0ucmlkZ2UobG9nKHByaWNlKSB+IChsb2coTG90LkFyZWEpICsgTmVpZ2hib3Job29kICsgQ29uZGl0aW9uLjEgKyAKICAgIE92ZXJhbGwuUXVhbCArIE92ZXJhbGwuQ29uZCArIEhvdXNlQWdlICsgRm91bmRhdGlvbiArIFRvdGFsLkJzbXQuU0YgKyAKICAgIENlbnRyYWwuQWlyICsgbG9nKFgxMi5TRikgKyBCYXRocyArIEtpdGNoZW4uQWJ2R3IgKyBLaXRjaGVuLlF1YWwgKyAKICAgIEZ1bmN0aW9uYWwgKyBGaXJlcGxhY2VzICsgR2FyYWdlLkNhcnMgKyBQYXZlZC5Ecml2ZSleMiArIEJzbXQuWU46QnNtdC5FeHBvc3VyZSwgCiAgICAgICAgICAgICAgICAgICAgICAgICBsYW1iZGEgPSBzZXEoMCwgNSwgMC4wMDAxKSwgZGF0YSA9IGRhdGEpIAoKZGF0YTIgPSBkYXRhICU+JSBtdXRhdGUoYnNtdC5xdWFsID0gQnNtdC5ZTipCc210LlF1YWwsIGJzbXQuY29uZCA9IEJzbXQuWU46QnNtdC5Db25kLAogICAgICAgICAgICAgICAgICAgICAgICBic210LmV4cG9zdXJlID0gQnNtdC5ZTjpCc210LkV4cG9zdXJlKQoKbG9hZCgifi9EZXNrdG9wL1NUQSA1MjEvUHJvamVjdF9ob21vQmF5ZXNpYW5zL2FtZXNfdGVzdC5SZGF0YSIpCmN0ZXN0ID0gYW1lc190ZXN0CmN0cmFpbiA9IGRhdGEKCiMgcmlkZ2U6CmhvdXNpbmcucmlkZ2UgPSBsbS5yaWRnZShsb2cocHJpY2UpIH4gKE1TLlN1YkNsYXNzICsgTVMuWm9uaW5nKyBsb2coTG90LkZyb250YWdlKSArIGxvZyhMb3QuQXJlYSkgKyBTdHJlZXQpXjIsCiAgICAgICAgICAgICAgICAgICAgICAgICBsYW1iZGEgPSBzZXEoMCwgNSwgMC4wMDAxKSwgZGF0YSA9IGRhdGEpIAoKYmVzdC5sYW1iZGEgPSBob3VzaW5nLnJpZGdlJGxhbWJkYVt3aGljaC5taW4oaG91c2luZy5yaWRnZSRHQ1YpXQoKCgojIGxhc3NvOgojIFggYW5kIFk6ClggPSBtb2RlbC5tYXRyaXgobG9nKHByaWNlKX4gKGxvZyhMb3QuQXJlYSkgKyBOZWlnaGJvcmhvb2QgKyBDb25kaXRpb24uMSArIAogICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBGb3VuZGF0aW9uICsgVG90YWwuQnNtdC5TRiArIAogICAgQ2VudHJhbC5BaXIgKyBsb2coWDEyLlNGKSArIEJhdGhzICsgS2l0Y2hlbi5BYnZHciArIEtpdGNoZW4uUXVhbCArIAogICAgRnVuY3Rpb25hbCArIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIFBhdmVkLkRyaXZlKV4yICsgQnNtdC5ZTjpCc210LkV4cG9zdXJlLCAKICAgICAgICAgICAgICAgICBkYXRhID0gZGF0YSkKWSA9IGFzLm1hdHJpeChkYXRhWywzXSkKCiMgbGFzc286Cmxhc3NvLmN2ID0gY3YuZ2xtbmV0KHggPSBYLCB5ID0gWSwgYWxwaGE9MSwgbGFtYmRhID0gc2VxKDAsIDIsIDAuMDAxKSwgZmFtaWx5ID0gImdhdXNzaWFuIikKbGFtYmRhLmxhc3NvID0gbGFzc28uY3YkbGFtYmRhLm1pbgojIGxhc3NvIHdpdGggdGhlIGJlc3QgbGFtYmRhCmhvdXNpbmcubGFzc28gPSBnbG1uZXQoWCwgWSwgYWxwaGE9MSwgbGFtYmRhID0gbGFtYmRhLmxhc3NvLCBmYW1pbHkgPSAiZ2F1c3NpYW4iKSAKY29lZihob3VzaW5nLmxhc3NvKQpgYGAKCgpgYGB7cn0KbGlicmFyeShtb25vbXZuKQojU0NBTEVEPz8KCgpYID0gbW9kZWwubWF0cml4KGxvZyhwcmljZSl+IGxvZyhMb3QuQXJlYSkgKyBhcy5udW1lcmljKE5laWdoYm9yaG9vZCkgKyBhcy5udW1lcmljKENvbmRpdGlvbi4xKSArIAogICAgT3ZlcmFsbC5RdWFsICsgT3ZlcmFsbC5Db25kICsgSG91c2VBZ2UgKyBhcy5udW1lcmljKEZvdW5kYXRpb24pICsgVG90YWwuQnNtdC5TRiArIAogICAgYXMubnVtZXJpYyhDZW50cmFsLkFpcikgKyBsb2coWDEyLlNGKSArIEJhdGhzICsgS2l0Y2hlbi5BYnZHciArIGFzLm51bWVyaWMoS2l0Y2hlbi5RdWFsKSArIAogICAgYXMubnVtZXJpYyhGdW5jdGlvbmFsKSArIEZpcmVwbGFjZXMgKyBHYXJhZ2UuQ2FycyArIGFzLm51bWVyaWMoUGF2ZWQuRHJpdmUpXjIgKyBhcy5udW1lcmljKEJzbXQuWU4pKmFzLm51bWVyaWMoQnNtdC5FeHBvc3VyZSksIAogICAgICAgICAgICAgICAgIGRhdGEgPSBkYXRhKQoKWC5zY2FsZWQgPSBzY2FsZSh1bmxpc3QoWCksIGNlbnRlciA9IFRSVUUsIHNjYWxlID0gVFJVRSkKWSA9IHVubGlzdChkYXRhWywgM10pCgpob3VzaW5nLmJsYXNzby5SSiA9IGJsYXNzbyhYLnNjYWxlZCwgWSwgUkogPSBUUlVFLCB2ZXJiPTApCnlwcmVkLmJsYXNzby5SSiA9IG1lYW4oaG91c2luZy5ibGFzc28uUkokbXUpICsgYXMubWF0cml4KHRlc3Quc2NhbGVkKSAlKiUgYXBwbHkoaG91c2luZy5ibGFzc28uUkokYmV0YSwgMiwgbWVhbikKc3VtbWFyeShob3VzaW5nLmJsYXNzby5SSikkY29lZgoKIyBwcmVkaWN0aW9uOgojIHlwcmVkLmJsYXNzby5SSiA9IG1lYW4oY29sbGVnZS5ibGFzc28uUkokbXUpICsgYXMubWF0cml4KHRlc3Quc2NhbGVkKSAlKiUgYXBwbHkoY29sbGVnZS5ibGFzc28uUkokYmV0YSwgMiwgbWVhbikKIyBybXNlLmJsYXNzby5SSiA9IHJtc2UoZXhwKGN0ZXN0JEFwcHMpLCBleHAoeXByZWQuYmxhc3NvLlJKKSkgCmBgYAoKCmBgYHtyfQojIGNlbnRlciBhbmQgc2NhbGUgWApYLnNjYWxlZCA9IFgKWC5zY2FsZWRbLDI6MTVdID0gc2NhbGUoWFssMjoxNV0sIGNlbnRlciA9Y29sbGVnZS5yaWRnZS5jdiR4bVtjKC0xLC0xNildLAogICAgICAgICAgICAgICAgICAgICAgICBzY2FsZSA9IGNvbGxlZ2UucmlkZ2UuY3Ykc2NhbGVzW2MoLTEsLTE2KV0pCiAgICAgICAgICAgICAgICAgICAgICAgIAojIFJKID0gRkFMU0UKIyBjb2xsZWdlLmJsYXNzbyA9IGJsYXNzbyhYLnNjYWxlZCwgWSwgUkogPSBGQUxTRSwgdmVyYj0wKSAKIyB5cHJlZC5ibGFzc28gPSBtZWFuKGNvbGxlZ2UuYmxhc3NvJG11KSArCiMgICAgICAgICAgICAgICAgYXMubWF0cml4KHRlc3Quc2NhbGVkKSAlKiUgYXBwbHkoY29sbGVnZS5ibGFzc28kYmV0YSwgMiwgbWVhbikgCiMgcm1zZS5ibGFzc28gPSBybXNlKGV4cChjdGVzdCRBcHBzKSwgZXhwKHlwcmVkLmJsYXNzbykpCiMgbW9kZWwgc2VsZWN0aW9uIHdpdGggUkogPSBUUlVFCmNvbGxlZ2UuYmxhc3NvLlJKID0gYmxhc3NvKFguc2NhbGVkLCBZLCBSSiA9IFRSVUUsIHZlcmI9MCkKeXByZWQuYmxhc3NvLlJKID0gbWVhbihjb2xsZWdlLmJsYXNzby5SSiRtdSkgKyBhcy5tYXRyaXgodGVzdC5zY2FsZWQpICUqJSBhcHBseShjb2xsZWdlLmJsYXNzby5SSiRiZXRhLCAyLCBtZWFuKQpybXNlLmJsYXNzby5SSiA9IHJtc2UoZXhwKGN0ZXN0JEFwcHMpLCBleHAoeXByZWQuYmxhc3NvLlJKKSkgCnN1bW1hcnkoY29sbGVnZS5ibGFzc28uUkopJGNvZWYKYGBgCgoKYGBge3J9Cm1vZGVsLmxhc3NvID0gZ2xtbmV0KFgudHJhaW4sIGxvZyhkYXRhX3RyYWluJHByaWNlKSwgYWxwaGE9MSkKY3YubGFzc28gPSBjdi5nbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0xKQp5aGF0Lmxhc3NvID0gZXhwKHByZWRpY3QobW9kZWwubGFzc28sIHM9Y3YubGFzc28kbGFtYmRhLm1pbiwgdHlwZT0icmVzcG9uc2UiLCBuZXd4ID0gWC50ZXN0KSkKCnNxcnQobWVhbigoKHloYXQubGFzc28pLWRhdGFfdGVzdCRwcmljZSleMikpCgoKIyBSaWRnZSBmb3IgdGhlIHNpbXBsaWZpZWQgbW9kZWwgKGZyb20gQklDKQoKbW9kZWwucmlkZ2UgPSBnbG1uZXQoWC50cmFpbiwgbG9nKGRhdGFfdHJhaW4kcHJpY2UpLCBhbHBoYT0wKQpjdi5yaWRnZSA9IGN2LmdsbW5ldChYLnRyYWluLCBsb2coZGF0YV90cmFpbiRwcmljZSksIGFscGhhPTApCnloYXQucmlkZ2UgPSBwcmVkaWN0KG1vZGVsLnJpZGdlLCBzPWN2LnJpZGdlJGxhbWJkYS5taW4sIHR5cGU9InJlc3BvbnNlIiwgbmV3eCA9IFgudGVzdCkKCnNxcnQobWVhbigoZXhwKHloYXQucmlkZ2UpLWRhdGFfdGVzdCRwcmljZSleMikpCmBgYAoKCgoKCgoKCiMjIyBQYXJ0IElJSTogV3JpdGUgVXAKCk9uY2UgeW91IGFyZSBzYXRpc2ZpZWQgd2l0aCB5b3VyIG1vZGVsLCBwcm92aWRlIGEgd3JpdGUgdXAgb2YgeW91ciBkYXRhIGFuYWx5c2lzIHByb2plY3QgaW4gYSBuZXcgUm1kIGZpbGUvcGRmIGZpbGU6IGB3cml0ZXVwLlJtZGAgYnkgY29weWluZyBvdmVyIHNhbGllbnQgcGFydHMgb2YgeW91ciBSIG5vdGVib29rLiBUaGUgd3JpdHRlbiBhc3NpZ25tZW50IGNvbnNpc3RzIG9mIGZpdmUgcGFydHM6CgoxLiBFeHBsb3JhdG9yeSBkYXRhIGFuYWx5c2lzICgyMCBwb2ludHMpOiBtdXN0IGluY2x1ZGUgdGhyZWUgY29ycmVjdGx5IGxhYmVsZWQgZ3JhcGhzIGFuZCBhbiBleHBsYW5hdGlvbiB0aGF0IGhpZ2hsaWdodCB0aGUgbW9zdCBpbXBvcnRhbnQgZmVhdHVyZXMgdGhhdCB3ZW50IGludG8geW91ciBtb2RlbCBidWlsZGluZy4KCgoyLiBEZXZlbG9wbWVudCBhbmQgYXNzZXNzbWVudCBvZiBhbiBpbml0aWFsIG1vZGVsIGZyb20gUGFydCBJICgxMCBwb2ludHMpCgoqIEluaXRpYWwgbW9kZWw6IG11c3QgaW5jbHVkZSBhIHN1bW1hcnkgdGFibGUgYW5kIGFuIGV4cGxhbmF0aW9uL2Rpc2N1c3Npb24gZm9yIHZhcmlhYmxlIHNlbGVjdGlvbi4gIEludGVycHJldGF0aW9uIG9mIGNvZWZmaWNpZW50cyBkZXNpcmFibGUgZm9yIGZ1bGwgcG9pbnRzLgoKKiBNb2RlbCBzZWxlY3Rpb246IG11c3QgaW5jbHVkZSBhIGRpc2N1c3Npb24KCiogUmVzaWR1YWw6IG11c3QgaW5jbHVkZSBhIHJlc2lkdWFsIHBsb3QgYW5kIGEgZGlzY3Vzc2lvbgoKKiBSTVNFOiBtdXN0IGluY2x1ZGUgYW4gUk1TRSBhbmQgYW4gZXhwbGFuYXRpb24gIChvdGhlciBjcml0ZXJpYSBkZXNpcmFibGUpCgoqIE1vZGVsIHRlc3Rpbmc6IG11c3QgaW5jbHVkZSBhbiBleHBsYW5hdGlvbgoKCgozLiBEZXZlbG9wbWVudCBvZiB0aGUgZmluYWwgbW9kZWwgKDIwIHBvaW50cykKCiogRmluYWwgbW9kZWw6IG11c3QgaW5jbHVkZSBhIHN1bW1hcnkgdGFibGUKCiogVmFyaWFibGVzOiBtdXN0IGluY2x1ZGUgYW4gZXhwbGFuYXRpb24KCiogVmFyaWFibGUgc2VsZWN0aW9uL3Nocmlua2FnZTogbXVzdCB1c2UgYXBwcm9wcmlhdGUgbWV0aG9kIGFuZCBpbmNsdWRlIGFuIGV4cGxhbmF0aW9uCgoKCjQuIEFzc2Vzc21lbnQgb2YgdGhlIGZpbmFsIG1vZGVsICgyNSBwb2ludHMpCgoqIFJlc2lkdWFsOiBtdXN0IGluY2x1ZGUgYSByZXNpZHVhbCBwbG90IGFuZCBhIGRpc2N1c3Npb24KCiogUk1TRTogbXVzdCBpbmNsdWRlIGFuIFJNU0UgYW5kIGFuIGV4cGxhbmF0aW9uICAob3RoZXIgY3JpdGVyaWEgZGVzaXJhYmxlKQoKKiBNb2RlbCBldmFsdWF0aW9uOiBtdXN0IGluY2x1ZGUgYW4gZXZhbHVhdGlvbiBkaXNjdXNzaW9uCgoqIE1vZGVsIHRlc3RpbmcgOiBtdXN0IGluY2x1ZGUgYSBkaXNjdXNzaW9uCgoqIE1vZGVsIHJlc3VsdDogbXVzdCBpbmNsdWRlIGEgc2VsZWN0aW9uIG9mIHRoZSB0b3AgMTAgdW5kZXJ2YWx1ZWQgYW5kIG92ZXJ2YWx1ZWQgaG91c2VzCgoKCjUuIENvbmNsdXNpb24gKDEwIHBvaW50cyk6IG11c3QgaW5jbHVkZSBhIHN1bW1hcnkgb2YgcmVzdWx0cyBhbmQgYSBkaXNjdXNzaW9uIG9mIHRoaW5ncyBsZWFybmVkCgoKCiMjIyBQYXJ0IElWCkNyZWF0ZSBwcmVkaWN0aW9ucyBmb3IgdGhlIHZhbGlkYXRpb24gZGF0YSBmcm9tIHlvdXIgZmluYWwgbW9kZWwgYW5kIHdyaXRlIG91dCB0byBhIGZpbGUgYHByZWRpY3Rpb24tdmFsaWRhdGlvbi5SZGF0YWAKVGhpcyBzaG91bGQgaGF2ZSB0aGUgc2FtZSBmb3JtYXQgYXMgdGhlIG1vZGVscyBpbiBQYXJ0IEkgYW5kIElJLgoKMTAgcG9pbnRzCgoKCiMjIyBDbGFzcyBQcmVzZW50YXRpb25zCgpFYWNoIEdyb3VwIHNob3VsZCBwcmVwYXJlIDUgc2xpZGVzIGluIHRoZWlyIEdpdGh1YiByZXBvOiAgKHNhdmUgYXMgc2xpZGVzLnBkZikKCiogTW9zdCBpbnRlcmVzdGluZyBncmFwaGljICAoYSBwaWN0dXJlIGlzIHdvcnRoIGEgdGhvdXNhbmQgd29yZHMgcHJpemUhKSAgCgoqIEJlc3QgTW9kZWwgKG1vdGl2YXRpb24sIGhvdyB5b3UgZm91bmQgaXQsIHdoeSB5b3UgdGhpbmsgaXQgaXMgYmVzdCkKCiogQmVzdCBJbnNpZ2h0cyBpbnRvIHByZWRpY3RpbmcgU2FsZXMgUHJpY2UuCgoqIDIgQmVzdCBIb3VzZXMgdG8gcHVyY2hhc2UgIChhbmQgd2h5KQoKKiBCZXN0IFRlYW0gTmFtZS9HcmFwaGljCgpXZSB3aWxsIHNlbGVjdCB3aW5uZXJzIGJhc2VkIG9uIHRoZSBhYm92ZSBjcml0ZXJpYSBhbmQgb3ZlcmFsbCBwZXJmb3JtYW5jZS4KCgpGaW5hbGx5IHlvdXIgcmVwbyBzaG91bGQgaGF2ZTogYHdyaXRldXAuUm1kYCwgYHdyaXRldXAucGRmYCwgYHNsaWRlcy5SbWRgIChhbmQgd2hhdGV2ZXIgb3V0cHV0IHlvdSB1c2UgZm9yIHRoZSBwcmVzZW50YXRpb24pIGFuZCBgcHJlZGljdC5SZGF0YWAgYW5kIGBwcmVkaWN0LXZhbGlkYXRpb24uUmRhdGFgLg==